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Chapter  1 


Summary 


1.1  Objectives 

The  objective  of  this  project  was  to  develop  and  demonstrate,  in  challenge  problems  selected  by 
DARPA  DSO,  techniques  for  managing  uncertainty  in  complex  dynamical  systems.  Out  of  the 
original  three-year  program,  the  first  two  phases  were  executed.  Each  of  these  phases  was  divided 
into  tool  development  as  well  as  meeting  challenge  problems  to  demonstrate  the  convergence  of 
these  tools  in  concerted  efforts. 

More  explicitly,  the  overarching  goal  of  the  project  was  to  develop  tools  and  workflows  for  quan¬ 
tifying  and  managing  uncertainty  in  ways  that  would  perform  orders  of  magnitude  faster  than 
Monte  Carlo  sampling  with  controlled,  provable  scaling  (preferably  linear  in  the  system  size).  The 
challenge  problems  were  designed  to  demonstrate  progress  toward  this  ultimate  goal. 

The  focus  of  Phase  I  was  to  show  that  the  techniques  selected  and  developed  could  be  applied 
correctly  to  systems  of  many  particles.  The  two  challenge  problems  for  this  phase  were: 

•  Self-assembly:  Obtain  an  interaction  potential  such  that  a  system  of  particles  in  a  box  would 
spontaneously  assemble  into  a  honeycomb  structure  and  compare  this  to  a  benchmark  solution 
from  the  literature  [9,  10]. 

•  Phase  diagram:  Obtain  the  phase  transition  temperature  of  a  noble  gas  physisorbed  on  a 
graphite  substrate,  demonstrating  that  the  team  could  correctly  extract  complex  emergent 
behavior  of  a  system  of  10,000  particles. 

Apart  from  the  further  development,  selection,  and  implementation  of  mathematical  tools  for  un¬ 
certainty  quantification,  Phase  II  included  the  following  challenge  problems: 

•  Phase  diagram  with  uncertainty:  Show  orders-of-magnitude  speed-up  over  Monte  Carlo  sam¬ 
pling  in  the  quantification  of  uncertainty  in  a  complex,  uncertain  system.  The  system  chosen 
for  this  challenge  was  a  monolayer  of  carbon  monoxide  (CO)  on  graphite  in  the  presence  of  an 


3 


i.2.  SUMMARY  OF  ACCOMPLISHMENTS 


Isotropic  potential  for  honeycomb 


Extension  to  anistropic  interactions 


Radtai  depenca  of 
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Figure  1.1:  Summary  of  main  results  for  the  self-assembly  problem. 


uneertain  level  of  argon  impurities.  The  system  exhibits  a  eomplex  phase  transition  [12]  and 
the  task  was  to  ealeiilate  the  transition  temperature,  ineluding  its  uneertainty,  as  a  funetion 
of  the  uneertain  argon  eoneentration. 

•  Surveillanee:  Design  seareh  strategies  for  50  simulated  unmanned  aerial  vehieles  (UAVs)  look¬ 
ing  for  a  stationary  target  in  a  complex  terrain  using  noisy  sensors  with  uncertain  footprint. 
The  strategics  had  to  exhibit  shorter  seareh  times  than  straightforward  lawnmow^er  patterns 
while  still  satisfying  constraints  on  detection  (lower  bound)  and  false  alarm  (upper  bound) 
probabilities. 


1.2  Summary  of  accomplishments 

The  performance  requirements  from  the  Phase  I  and  II  challenge  problems  were  met,  and  in  some 
cases  surpassed  by  orders  of  magnitude  above  the  required  acceleration.  Here  we  summarize  the 
accomplishments  directly  related  to  the  challenge  problems.  Chapter  2  summarizes  the  tools  de¬ 
veloped,  organized  by  themes. 

Phase  I  self-assembly  challenge 

•  As  shown  in  appendix  C.l  and  reference  [J3],  the  team  first  developed  several  relevant  metrics 
for  lattice  quality  and  then  applied  trend  optimization  using  ridge  regression  ([J3]  and  [11])  to 
obtain  solutions  superior  (in  terms  of  robustness  of  the  self-assembly)  to  the  benchmark  [10]. 
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The  most  robust  solutions  were  particularly  counterintuitive  in  that  the  resulting  interaction 
potential  was  purely  repulsive.  The  team  also  showed  that  extending  the  interaction  to 
anisotropic  potentials  can  yield  much  more  robust  self-assembly,  even  of  structures  that  were 
identified  in  the  Hterature  as  impossible  to  obtain  through  central  potentials  ([J9]  and  [10]). 

•  The  self-assembly  problem  also  exposed  the  team  to  the  state  of  the  art  in  molecular  dynamics 
simulation  algorithms  and,  in  particular,  to  the  different  approaches  for  simulation  of  systems 
of  particles  with  noise,  typically  due  to  contact  with  a  thermal  bath.  This  spurred  the 
development  of  a  completely  novel  approach  to  simulating  noisy  systems  that  preserve  the 
specific  structure  of  the  noise  in  a  controlled  manner  [J2].  In  other  words,  this  is  the  stochastic 
equivalent  of  variational  integrators,  with  the  difference  that  instead  of  preserving  energy  (as 
in  the  widely  used  Verlet  algorithm  [4])  they  preserve  the  invariant  measure. 

•  Other  mathematical  results  related  to  the  self-assembly  challenge  are  a  proof  that  central 
potentials  cannot  yield  certain  structures  when  the  system  is  not  confined  to  a  fixed-volume 
box  [J9]  and  several  provably-correct  metrics  for  quantifying  the  distance  between  simulation 
results  and  target  lattices  (appendix  C.2  and  [Jll]).  Finally,  several  tunable  lattice  quality 
measures  were  developed  (appendix  D.2  and  [J12]).  These  can  be  selected  to  emphasize 
different  desired  qualities  in  the  target  lattice  (e.g.,  shape  vs.  density)  and  can  be  used  for 
self-assembly  as  well  as  for  phase  diagram  computations. 

Figure  1.1  summarizes  the  main  results  for  the  self-assembly  challenge  problem. 

Phase  I  phase  diagram  challenge 

•  In  appendix  E.l  [J17]  the  team  developed  a  new  class  of  Hidden  Markov  Models,  the  finite- 
rank  optimal-prediction  (FRO)  model,  for  quickly  learning  the  dynamics  of  a  system.  This 
new  tool  was  used  to  learn  from  MD  simulations  of  helium  atoms  physisorbed  on  graphite  the 
dynamics  of  a  coarse  variable  relevant  to  the  phase  transition  (the  potential  energy  per  atom). 
The  transition  is  then  associated  with  metastability  in  the  spectrum  of  the  Markov  model. 
The  team  showed  that  it  is  faster  to  directly  learn  when  the  spectrum  exhibits  metastability 
than  to  directly  simulate  the  system  until  it  settles  into  its  stationary  distribution.  The 
method  was  later  extended  into  reference  [C4],  where  it  was  applied  to  fast  decentralized 
control  over  networks  through  the  construction  of  multiple  local  Markov  models. 

•  Model  order  reduction  was  approached  from  the  point  of  view  of  data  clustering  and  stochastic 
modeling.  A  Markov  matrix  whose  state  space  is  the  possible  size  of  clusters  can  be  learned 
from  the  simulation  of  molecular  systems  at  specific  conditions,  such  as  temperature,  density 
and  pressure  with  prior  belief.  The  expectation  value  of  an  invariant  distribution  of  learned 
Markov  matrix  indicates  the  phase  transition  of  the  molecular  dynamics  system  qualitatively 
while  the  second  largest  eigenvalue  modulus  can  be  used  as  a  quantitative  indicator.  As  a 
consequence,  the  stochastic  reduced  order  model  not  only  reduces  the  order  of  the  system 
based  on  the  choice  of  coarse  variable  but  also  provides  an  insight  of  macroscopic  properties. 
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Lise  o'  FeyrifTian  s  Quasi  Classical  Potential 
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ayreement  witfi  enpenment 
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with  RUM  tools 


KlO’  *10*  *10'  10* 
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Figure  1.2:  Summary  of  phase  I  results  for  the  phase  diagram  })roblem. 


Heuristic  graph  decomposition  Bayesian  estimation  is  shown  to  be  more  reliable  and  robust 
than  maximum  likelihood  estimation  because  it  reflects  prior  information  on  the  system. 

•  Two  orders  of  magnitude  acceleration  over  Molecular  Dynamics  10,000  atom  baseline  was 
achieved  by  running  a  reduced  size  system  of  only  100  particles.  To  assess  the  error  in  the 
phase  transition  temperature  determined  using  lower  number  of  atoms  in  MD  simulation, 
convergence  of  the  phase  transition  temperature  was  studied  using  numerical  and  analytical 
methods. 

•  In  reference  [Jl]  the  team  builds  on  the  Coarse  Molecular  Dynamics  technique  [1]  to  obtain 
the  order-to-disorder  transition  temperature  of  krypton  physisorbed  on  graphite.  The  team 
obtains  5x  acceleration  compared  to  standard  MD  measurements  of  fluctuations  of  the  to¬ 
tal  energy.  The  CMD  technique  falls  under  the  umbrella  of  the  more  general  equation-free 
methods,  in  which  the  macroscopic  evolution  of  a  system  is  simulated  by  doing  short  bursts 
of  microscopic- level  simulations  compatible  with  the  required  macroscopic  state.  Initializ¬ 
ing  such  microscopic  systems  is  called  lifting,  and  doing  it  efficiently  is  an  area  of  active 
development  (see  appendix  B.7  and  [J8]). 

•  A  parallel  effort  for  the  krypton  problem  (appendix  D.l)  was  to  extend  the  use  of  quenching 
simulations  [5]  by  developing  pattern  boundary  detection  methods  to  separate  high  and  low 
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10000  CO  (WKcutM 


Figure  L3:  Summary  of  Phase  II  results  for  the  phase  diagram  problem.  From  left  to  right:  system 
snapshot  showing  pinwheel  structures  around  argon  impurities  (blue);  flip  moves  used  in  Ising-type 
model;  heat  capacity  curves  for  different  impurity  concentrations;  acceleration  in  calculation  of  the 
transition  temperature  for  Monte  Carlo  (blue)  and  PCM  (red). 


density  regions  that  appear  spontaneously  in  first  order  phase  transitions. 

Figure  1.2  summarizes  these  results.  Note  that,  in  order  to  obtain  a  good  match  with  the  experi¬ 
mental  results  from  the  literature  for  the  case  of  helium,  the  team  had  to  add  a  quantum-mechanical 
correction  to  the  classical  potential  used  in  the  simulations.  This  correction,  based  on  Feynman’s 
quasi-classical  potential,  went  beyond  the  asymptotic  approximations  from  [14]  that  were  used  in 
the  theoretical  calculations  of  reference  [2]. 

Phase  II  phase  diagram  with  uncertainty  challenge  Appendix  D.3  [J21]  focuses  on  the  low 
temperature  phase  transition  for  carbon  monoxide  (CO)  physisorbed  on  a  graphite  substrate. 

•  The  team  first  developed  an  Ising-type  model  for  the  system  that  accurately  captures  the 
phase  transition  in  the  presence  of  an  uncertain  concentration  of  argon  impurities. 

•  Since  in  the  simulations  the  number  of  argon  impurities  had  to  be  an  integer,  the  team 
had  to  extend  the  Polynomial  Chaos-based  Probabilistic  Collocation  Method  [3]  to  the  case 
where  the  uncertain  parameters  can  only  take  integer  values,  leading  to  a  rare  application  of 
Krawtchoiik  polynomials.  PCM  allowed  the  team  to  calculate  the  mean  and  variance  of  the 
phase  transition  temperature  2000  times  faster  than  Monte  Carlo  sampling  [J21]. 

•  The  team’s  result  settled  the  scientific  question  of  whether  the  ground  state  of  CO  on  graphite 
is  head-to-head  ordered  or  head-to-tail  ordered  in  favor  of  the  latter.  Furthermore,  the  team 
showed  that  formation  of  pinwheel  regions  of  CO  around  argon  atoms  are  at  the  origin  of  the 
anomalous  effect  of  stabilization  of  the  low- temperature  phase  [J21]. 

Figure  1.3  shows  the  system,  a  typical  configuration  in  the  Ising-type  model,  a  representative 
flip  move  (molecule  rotation)  in  the  computational  procedure,  the  variations  in  the  specific  heat 
vs.  temperature  curves,  and  the  comparative  acceleration  of  PCM  over  Monte  Carlo. 


7 


1.2.  SUMMARY  OF  ACCOMPLISHMENTS 


Phase  II  surveillance  challenge  Appendix  A  as  well  as  references  [J4,  J6,  Jl3,  J18,  Jl9], 
[C5],  and  [6]  contain  extended  reports  on  results  relevant  to  the  surveillance  challenge  problem. 
Figure  1.4  shows  the  search  area  used  by  the  team  to  compare  the  performance  of  the  algorithms 
developed  in  the  project  with  that  of  both  standard  and  “smart”  lawnmower  search  patterns  that 
take  into  account  the  prior  distribution  for  the  single  target.  The  challenge  included  the  following 
constraints^  so  as  to  make  the  different  algorithms  comparable: 

•  There  is  a  single,  immobile  target  to  be  found. 

•  The  vehicles’  sensors  have  a  small  footprint  compared  to  the  total  search  area  (0.1%  for 
one  sensor,  5%  for  the  whole  swarm)  and  their  dynamics  must  be  constrained  (speed  and 
acceleration  limits). 

•  The  sensors  are  noisy:  at  each  observation,  a  sensor  has  a  probability  of  detection  <  1  and 
false  alarm  Sfa  >  0. 

•  The  terrain  includes  foliage.  If  the  target  is  in  the  foliage,  it  is  undetectable.  The  algorithms 
must  be  able  to  conclude  that  the  target  is  undetectable  after  a  finite  time. 

•  The  algorithms  as  a  whole  must  exhibit  a  global  probability  of  detection  above  a  given  thresh- 
old  Pd, global  and  false  alarm  rate  below  a  given  threshold  Pfa, global- 

Under  these  constraints,  the  algorithms  compete  for  lowest  median  detection  time. 

As  shown  in  figure  1.4,  two  different  lines  of  attack  yielded  successful  practical  search  strategies: 
Spectral  Multiscale  Search  (SMS),  Greedy  Spirals,  and  Dynamic  Greedy  Search  (DyGS).  Each  uses 
a  very  different  approach  and  has  its  own  strengths.  These  strengths  treat  different  axes  of  the 
problem  and  could  in  the  future  be  combined  into  a  unified  approach  to  control  a  swarm  of  UAVs 
performing  autonomous  search  missions.  Both  methods  achieved  almost  2x  reduction  in  median 
search  time  compared  with  smart  lawnmower. 

Spectral  Multiscale  Search,  or  SMS  (appendix  A.l  and  [J13]),  which  combines  a  novel  application 
of  the  Neyman-Pearson  lemma  [J6]  with  a  Lyapunov  method,  is  a  fully-autonomous  approach 
that  flexibly  dictates  the  required  control  forces  on  the  whole  swarm  at  every  time  step.  Given 
the  prior  distribution  for  the  single  target,  the  method  evaluates  how  much  the  time- integrated 
coverage  differs  from  the  prior,  using  a  specially-designed  weighted  measure  that  yields  a  naturally 
multiscale  approach.  The  method  spontaneously  spreads  out  the  vehicles,  initially  covering  the 
large-scale  features  of  the  prior  and  then  filling  in  the  smaller  scale  details.  As  shown  in  figure  1.4, 
this  method  can  take  into  consideration  both  uniform  and  nonuniform  priors.  The  vehicles  avoid 
the  foliage  when  possible,  but  spontaneously  fly  over  it  when  needed  to  cover  a  different  region. 

Figure  1.4  also  shows  the  ROC  (Receiver  Operating  Characteristic)  curves  associated  with  the  SMS 
decision  algorithm  [J13].  For  a  given  sensor  quality  (i.e.,  for  given  parameters  Sd  and  5fa)  these 
curves  show  graphically  the  effect  of  taking  repeated  measurements  in  an  area  and  help  determine 
how  much  coverage  is  needed  before  the  global  constraints  Pd, global  and  Pfa, global  are  satisfied. 

Dynamic  Greedy  Search,  or  DyGS  (appendix  A. 2  and  [J4]),  is  made  of  two  parts:  a  grid-free  decision 
algorithm  and  a  trajectory  planner.  The  trajectory  planner  is  based  on  a  specially-developed 
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Extensions 

V 


700' 


Search  area:  foliage  (gray),  prior  (red) 

5000  targets  (3057  urgets  not  in  foliage) 


:oo  400 


800  1000  1200  1400 


Baseline  solution:  lawnmower 


Spectral  Multiscale  Search 
(UCSB/AlMdyn) 


Improvements 

Greedy  Spirals  (Caltech) 


.9x  faster 


2.2x  faster 

Dynamic  Greedy  Search 
(Caltech) 

1.7x  faster 


ROC  curves 


Ft.  Benning  simulation 


Figure  1.4:  Top  left;  search  area  used  for  the  simulations.  The  gray  zones  mark  the  foliage  and 
the  shades  of  red  mark  the  prior  distribution  for  the  target.  The  dots  mark  5000  target  positions 
sampled  from  the  prior.  Also  shown  is  the  lawnmower  baseline  and  the  improved  algorithms  SMS, 
Greedy  Spirals,  and  DyGS  (see  text).  The  bottom  shows  an  approach  combining  helicopter  model 
trajectory  segments  and  roadmap  planning  for  near- real  time  control  of  UAVs  over  Fort  Benning. 
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method  for  optimizing  dynamically  the  path  to  take  in  a  computationally  tractable  manner  while 
still  producing  realizable  trajectories.  It  is  based  on  the  use  of  a  library  of  elementary  trajectory 
segments  that  individually  satisfy  the  vehicle  dynamics  and  can  be  interlocked  to  produce  large 
scale  roadmaps  (see  also  section  2.3.1). 

Even  though  the  challenge  problems  focused  on  the  detection  of  a  single  target,  the  team  identified 
that  the  need  to  detect  multiple  targets  would  naturally  arise  in  the  future,  and  the  necessary 
mathematics  to  treat  this  problem  in  a  computationally-efficient  manner  had  to  be  developed. 
Appendix  A. 4  (see  also  [C5])  develops  such  tools.  The  approach  is  novel  in  that  it  makes  efficient 
a  computation  that  in  its  original  form  is  computationally  intractable. 


1.3  Organization  of  the  report 

The  summary  of  accomplishments  presented  above  focuses  on  the  convergence  of  the  different  tools 
selected  and  developed  as  applied  to  the  solution  of  the  Phase  I  and  Phase  II  challenge  problems. 
In  contrast,  chapter  2  summarizes  other  tools  that  were  mostly  preparation  for  Phase  III  and  were, 
for  the  most  part,  not  used  directly  in  the  solution  of  the  Phase  I  and  Phase  II  challenge  problems. 

For  published  results  we  include  the  reference  to  the  appropriate  journal  or  conference  proceedings, 
while  for  results  that  are  submitted  or  in  preparation  we  include  the  full  drafts  or  internal  reports  as 
appendices.  The  appendices  are  mostly  self-contained,  and  as  such  the  citations  in  the  appendices 
refer  to  their  own  bibliographies,  and  not  to  the  report’s  main  bibliography. 
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Chapter  2 


Other  tools  developed 


This  project  deals  with  the  quantification  and  robust  management  of  uncertainty  in  complex  sys¬ 
tems.  In  view  of  the  goal  of  a  real-time  demonstration,  many  tools  were  developed  with  the  goal 
of  supporting  fast,  decentralized  analysis  of  the  situation,  as  well  as  efficient  design  of  vehicle 
trajectories  compatible  with  complex  dynamics. 

In  this  chapter  we  describe  some  of  the  tools  developed.  As  these  focused  on  improving  different 
aspects  of  the  problem,  the  tools  are  very  heterogeneous,  and  fall  broadly  in  the  following  three 
categpries: 

•  Graph  theoretic  methods  for  system  analysis  and  uncertainty  quantification 

•  Decentralized  estimation 

•  Design  of  dynamics 

2.1  Graph  theoretic  methods  for  system  analysis  and  uncertainty 
quantification 

In  order  to  make  a  system  of  many  interacting  components  tractable,  the  system  must  be  analyzed 
and  divided  into  weakly  connected  components  such  that  each  individual  component  is  of  reasonable 
size.  However  the  automatic  detection  of  such  components  is  a  difficult  task.  Appendices  B.5 
and  B.6  [J14,  J15]  focus  on  the  analysis  of  such  systems  using  graph-theoretic  techniques  to  obtain 
both  weak  connections  as  well  as  causal  chains.  The  latter  are  important  for  predicting  the  flow  of 
information,  and  therefore  uncertainty  propagation,  through  such  a  system. 

Similar  methods  were  applied  in  reference  [C3],  where  waveform  relaxation  was  used  to  simulate 
a  system  with  weakly  connected  components.  This  method  was  further  extended  in  [J24]  (see 
appendix  B.l)  into  a  unified,  scalable  approach  to  uncertainty  quantification.  Further  uncertainty 
quantification  techniques  are  compared  in  a])pendix  B.2 
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Another  use  of  graph-theoretic  techniques  is  the  application  of  diffusion  maps  to  detect  slow  vari¬ 
ables  in  a  system  [J22].  Once  such  slow  variables  have  been  found,  an  equation- free  approach  can 
be  used  to  accelerate  the  simulation  of  the  system  (see  [Jl,  J8]  and  [1,  13,  15]). 

Finally,  reference  [J23]  uses  the  connectivity  of  a  network  to  do  global  filtering  of  noisy  measure¬ 
ments. 


2.2  Decentralized  estimation 

In  anticipation  of  future  challenges  where  many  vehicles  are  collaborating  in  an  environment  with 
limited  communication  (or,  equivalently,  to  extend  life  in  power- limited  wireless  networks),  several 
tools  were  developed  to  deal  with  decentralized  estimation. 

•  For  large  networks  and,  in  particular,  for  networks  where  the  connectivity  is  changing  dy¬ 
namically),  stochastic  multiscale  consensus  was  developed  [C2].  Here  each  node  decides  at 
random  whether  to  simply  pa>ss  along  a  measurement  received  from  a  different  node  or  to  do 
a  computation  on  it.  The  act  of  passing  information  along  produces  long  scale  connections 
that  accelerate  the  convergence  of  consensus  algorithms. 

•  For  the  problem  of  searching  for  a  target,  appendix  B.3  deals  with  the  issue  of  several  different 
noisy  sensors  having  to  make  a  decision  as  to  whether  the  target  has  been  detected  based  on 
the  limited  information  they  have  shared  up  to  that  point. 

2.3  Design  of  dynamics  tools 

The  area  of  design  of  dynamics  encompasses  both  the  design  of  systems  that  will  spontaneously 
behave  in  a  desired  way  as  well  as  simplifying  computations  on  complex  systems  so  the  problem  of 
assigning  tasks  becomes  tractable. 

Examples  of  design  of  systems  that  spontaneously  l:>eliave  as  desired  are  given  in  appendix  C,  where 
the  self-assembly  tools  are  described  in  detail.  A  related  problem  is  that  of  targeted  activation  [Cl], 
where  the  dynamics  of  the  system  are  exploited  to  minimize  the  required  energy  input  to  obtain 
global  reconfigurations. 

For  simplifying  computations  in  systems  with  many  independent  actuators,  a  good  example  is  the 
fast  reconstruction  of  wavefronts  for  telescopes  with  adaptive  optics  (see  [8]  and  [7]  for  experimental 
validation).  Also,  see  appendix  B.4,  where  a  connection  to  the  problem  of  self- localization  is  made. 

2.3.1  Control  optimization  of  vehicles  with  obstacle  avoidance 

With  the  goal  of  developing  efficient  methods  to  control  vehicles  with  complex  dynamics  in  en¬ 
vironments  with  obstacles,  a  general  framework  was  developed  for  integrating  the  dynamics  and 
optimizing  the  motions  of  mechanical  systems.  The  resulting  algorithms  are  superior  to  standard 
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methods  in  numerical  robustness  and  effciency^  and  can  be  applied  to  many  types  of  vehicles  such 
as  simple  helicopters  and  hovercraft. 

The  general  approach  is  based  on  a  combination  of  standard  optimal  control  techniques  and  classical 
search  and  dynamic  programming  methods.  These  methods  stand  on  top  of  a  robust  numerical 
representation  of  the  underlying  vehicle  dynamics  derived  using  the  theory  of  discrete  mechanics. 
The  main  results  can  be  summarized  as: 

•  structure-respecting  geometric  discretization  of  mechanical  systems  with  symmetries,  internal 
actuated  shape,  and  nonholonomic  constraints 

•  discrete  optimal  control  formulation  that  respects  the  geometric  structure 

•  combining  the  derived  local  optimal  control  techniques  with  global  search  methods  in  order 
to  guarantee  near-globally  optimal  solutions 

•  extending  the  basic  motion  planning  framework  to  handle  more  specific  tasks  such  as  time- 
varying  goal  state,  maximizing  sensor  coverage,  deploying  multiple  vehicles  to  maximize  in¬ 
formation  about  a  goal  with  uncertain  dynamics  multiple  vehicles 
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tion.  In  preparation,  2008. 
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preparation,  2009. 
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robust  ergodic  search  algorithms.  In  preparation.  UCSB  confidential  preprint,  2008. 

[J19]  S.  Nair  and  J.  E.  Marsden.  Collision  avoidance  and  surveillance  with  autonomous  vehicles. 
In  preparation,  2009. 

[J20]  S.  Nair,  S.  Ober-Blobaiim,  and  J.  E.  Marsden.  The  Jacobi-Maiipertuis  principle  in  varia¬ 
tional  integrators.  In  preparation,  2009. 
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phase  in  carbon  monoxide  monolayers  on  graphite.  To  appear  in  Phys.  Rev.  B,  2009. 
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in  stochastic  dynamical  systems  by  anisotropic  diffusion  maps.  Submitted  to  Proc.  Nat. 
Acad.  Sci.,  2009. 

[J23]  A.  Singer,  Y.  Shkolnisky,  and  B.  Nadler.  Diffusion  interpretation  of  nonlocal  neighborhood 
filters  for  signal  denoising.  SIAM  J.  Imaging  Sci.,  2:118-139,  2009. 

[J24]  A.  Surana  and  A.  Banaszuk.  Scalable  uncertainty  quantification  in  complex  dynamic  net¬ 
works.  In  preparation,  2009. 
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[Cl]  B.  Eisenhower  and  I.  Mezic.  Actuation  requirements  in  high-dimensional  oscillator  systems. 
In  Proceedings  of  the  American  Control  Conference,  Seattle,  2008. 
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[C2]  J.-H.  Kim,  M.  West,  S.  Lall,  E.  Scholte,  and  A.  Baiiaszuk.  Stochastic  miiltiscale  approaches 
to  consensus  problems.  In  Proceedings  of  the  47th  IEEE  Conference  on  Decision  and  Control^ 
pages  5551-5557,  Cancun,  Mexico,  2008. 

[C3]  G.  Mathew,  S.  P.  Meyn,  and  A.  Banaszuk.  Waveform  relaxation  and  graph  decomposition. 
In  Proceedings  of  the  18th  International  Symposium  on  Mathematical  Theory  of  Networks 
and  Systems^  Blacksburg,  Virginia,  2008. 

[C4]  S.  P.  Meyn  and  G.  Mathew.  Shannon  meets  Bellman:  Feature  based  Markovian  models  for 
detection  and  optimization.  In  Proceedings  of  the  47th  IEEE  Conference  on  Decision  and 
Control,  pages  5558-5564,  Cancun,  Mexico,  2008. 

[C5]  S.  Nair,  K.  Reddy,  H.  Owhadi,  and  J.  Marsden.  Multitarget  detection  using  Bayesian  learn¬ 
ing.  Submitted  to  the  48th  IEEE  Conference  on  Decision  and  Control,  2009. 

4.3  Invited  sessions 

The  following  invited  sessions  and  minisymposia  were  organized  with  AFOSR  support  and  contain 

AFOSR-funded  papers: 

•  SIAM  Conference  on  Applications  of  Dynamical  Systems  2007.  Minisymposia:  Uncertainty 
quantification  in  large-scale  dynamical  systems.  Parts  I  and  II. 

•  Mathematical  Theory  of  Networks  and  Systems  2008.  Special  session  on  robust  uncertainty 
management. 

•  Allerton  Conference  on  Communication,  Control,  and  Computing  2008.  Session:  Optimiza¬ 
tion  and  learning. 

•  47th  IEEE  Conference  on  Decision  and  Control  2008.  Session:  Complex  systems:  multiplayer 
models. 

•  SIAM  Conference  on  Applications  of  Dynamical  Systems  2009.  Minisymposia:  Uncertainty 
quantification  of  high-diniensional  random  dynamical  systems,  Parts  I  and  II. 

•  SI  AM  Conference  on  Applications  of  Dynamical  Systems  2009.  Minisymposium:  Uncertainty 
management  and  trajectory  plannning  in  dynamic  multi-agent  surveillance  systems. 

•  SIAM  Conference  on  Applications  of  Dynamical  Systems  2009.  Minisymposium:  Graph- 
theoretic  methods  for  analysis  of  complex  networks. 

•  48th  IEEE  Conference  on  Decision  and  Control  2009.  Sessions:  Learning  and  control,  Parts 
I  and  II. 


17 


4.4.  PLENARY,  KEYNOTE  AND  INVITED  TALKS 


4.4  Plenary,  Keynote  and  Invited  talks 

•  2007  SIAM  conference  of  Control  Theory,  San  Francisco  CA.  Plenary  talk,  A.  Banaszuk. 

•  2008  Stanford  Structured  Integrator  Workshop.  H.  Owhadi. 

•  Dynamics  Days  2008,  Knoxville,  Tennessee.  Invited  talk,  A.  Banaszuk. 

•  Lecture  series  on  networks  and  complex  systems.  Lunteren  Conference  on  the  Mathematics 
of  Operations  Research,  January  13-15,  2009.  S.  R  Meyn. 

•  SIAM  Conference  on  Applications  of  Dynamical  Systems  2009,  Snowbird  UT.  Plenary  talk, 
Igor  Mezic. 
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Abstract 

VVe  present  a  search  algorithm  for  single  or  multiple  searchers  that  finds  a  stationary  target 
in  presence  of  uncertainty  in  sensor  radius.  The  considered  uncertainty  condition  simulate  the 
influence  of  the  changing  environment  that  occur  in  practical  applications.  Uncertainty  in  sensor 
radius  sets  this  problem  apart  from  the  usual  search  and  surveillance  problem  setting.  Given  Pp 
and  Pfa^  the  algorithm  minimizes  search  time  to  find  the  target  with  probability  of  detection  at 
least  Pd  and  probability  of  false  alarm  at  most  Pfa>  We  prove  that  the  algorithm  discovers  the 
target  with  the  desired  efficiency.  Computer  simulations  show  that  our  algorithm  has  excellent 
performance  when  compared  with  Billiard  search  which  is  a  type  of  random  search.  Form  the 
design  of  the  algorithm,  it  follows  that  the  search  time  is  inversely  proportional  to  the  number  of 
searchers  participating. 


1  Introduction 

Study  of  search  problems  as  formalized  mathematical  models  started  more  than  60  years  ago,  for  a 
survey  see  [1].  During  World  War  II  mathematical  theory  was  applied  for  the  first  time  to  locate  Ger¬ 
man  submarine  threats  in  the  Atlantic.  Since  its  first  applications  search  theory  developed  somewhat 
detached  from  practical  applications.  Our  theory  stands  out  from  this  trend  because  it  uses  realistic 
dynamics  to  model  movement  of  searchers  and  in  addition  it  is,  apparently,  the  first  model  in  literature 
that  incorporates  uncertainty  in  sensor  radius  that  is  a  significant  factor  that  affects  search  missions  in 
real  life.  An  extensively  studied  setting  that  is  similar  to  ours,  see  [2],  [6],  is  when  a  target  is  located 
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somewhere  in  a  region  that  is  partitioned  into  a  number  of  cells.  The  probability  distribution  for 
the  targets  position  (i.e.,  the  probability  that  the  target  is  in  any  particular  cell),  and  the  detection 
function  of  our  sensor  (i.e.,  the  probability  of  detection  versus  effort  spent  searching  a  cell,  given  that 
the  target  resides  in  that  cell)  are  given.  The  goal  is  to  maximize  the  probability  of  detection  of 
the  target,  assumed  that  amount  of  total  effort  available  for  the  search  is  fixed.  A  major  drawback 
of  this  problem  is  its  discrete  setup,  that  would  require  perfectly  functioning  sensors.  Besides,  the 
theoretical  solutions  given  to  this  problem  assume  that  the  search  effort  is  infinitely  divisible  between 
cells  and  result  in  trajectories  that  would  be  physically  hard  to  follow.  Recently,  several  application 
oriented  algorithms  have  been  developed  for  similar  problems.  [8]  presents  a  receding- horizon  cooper¬ 
ative  search  algorithm  that  jointly  optimizes  routes  and  sensor  orientations  for  a  team  of  autonomous 
agents  searching  for  a  mobile  target.  The  algorithm  in  [8]  reduces  the  continuous  search  problem 
to  an  optimization  on  a  finite  graph.  In  [11]  a  framework  for  cooperative  search  using  UAV  swarms 
is  described.  The  algorithm  in  [11]  sweeps  the  area  with  UAVs  flying  side-by-side  in  straight  lines. 
Unfortunately,  both  algorithms  of  [8]  and  [11]  do  not  take  into  account  changes  in  the  environment 
that  may  occur.  The  changing  environment  (such  as  wind  or  fog)  may  alter  the  effective  radius  of 
the  sensor.  This  would  lead  to  leaving  parts  of  the  area  completely  uncovered  and  would  reduce  the 
performance  of  the  search  algorithm.  We  consider  the  search  problem  where  a  stationary  target  is 
placed  in  an  area  A  that  contains  foliage  F  that  the  sensors  can  not  penetrate.  We  consider  the  a 
priori  distribution  of  the  location  of  the  target  known  (if  it  is  not  given  we  assume  it  to  be  the  uniform 
distribution).  The  searchers  move  through  the  area  A  in  continuous  motion  and  use  a  circular  sensor 
to  scan  the  area.  Our  goal  is  to  minimize  search  time  in  the  presence  of  uncertainty  in  sensor  radius 
while  keeping  the  probability  of  detection  of  our  algorithm  above  threshold  Pp  and  probability  of 
false  alarm  of  the  algorithm  below  threshold  Pfa-  In  our  Spectral  Multi  Scale  (SMS)  algorithm  we 
utilize  the  Ney man- Pearson  lemma,  that  is  central  in  binary  hypothesis  testing  theory,  to  design  the 
decision  making  rule,  that  allows  the  searchers  to  quickly  locate  target  suspects  as  they  cover  the  area. 
The  algorithm  puts  some  of  the  searchers  in  rechecking  mode  to  take  some  additional  measurements 
at  target  suspects  positions.  This  strategy  ensures  that  the  probability  of  false  alarm  is  within  the 
required  threshold.  We  use  the  //“*  coverage  strategy  described  in  Section  3  to  cover  the  area  A. 
We  tested  the  SMS  algorithm  with  50  searchers  for  different  a-priori  target  distributions,  each  time 
making  5000  independent  simulations.  Our  computer  simulations  show  that  besides  demonstrating 
superior  robustness  in  presence  of  uncertainty  the  SMS  search  vastly  outperforms  Billiard  search  when 
searchers  start  out  in  random  directions  and  move  in  straight  lines,  reflecting  when  they  reach  the 
border.  The  median  absolute  deviation  of  SMS  search  time  is  1.5  times  smaller  than  that  of  Billiard 
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search;  median  search  time  of  SMS  search  is  1.6  times  smaller  than  that  of  Billiard  search;  median 
detection  time  of  SMS  search  1.7  times  smaller  than  that  of  Billiard  search.  Another  important  ad¬ 
vantage  of  the  SMS  algorithm  its  effective  use  of  assets:  the  search  time  is  inversely  proportional  to 
the  number  of  searchers.  So,  for  example,  if  we  have  two  searchers  instead  of  one,  the  expected  search 
time  is  half  of  what  we  would  expect  with  one  searcher. 


2  The  decision  making  strategy 

Let  us  consider  the  problem  where  N  searchers  are  moving  inside  a  search  area  A  in  with  the 
objective  to  detect  a  point-like  target.  We  assume  that  each  searcher  has  a  circular  sensor  with  radius 
at  most  S.  We  will  consider  various  scenarios  for  uncertainty  in  sensor  radius.  The  target  can  be  either 
in  the  search  area  A  or  in  the  foliage  F  where  the  searcher  can  not  detect  it.  We  assume  that  with 
probability  a  the  target  is  in  F  and  with  probability  1  —  a  the  target  is  \n  S  =  A  \  F.  We  assume 
that  the  probability  distribution  of  location  of  the  target  is  known.  The  probability  of  detection  for 
a  single  measurement,  3^,  is  the  probability  of  getting  a  reading  1  on  our  sensor,  assuming  that  the 
target  is  within  the  sensing  area.  The  probability  of  false  alarm  for  a  single  measurement,  sya,  is  the 
probability  of  getting  a  reading  1,  assuming  that  the  target  is  not  within  the  sensing  area.  Note,  that 
for  any  sensor  >  Sfa-  The  studies  on  real-life  sensors  indicate  that  as  Sd  increases,  so  does  s/q. 

We  denote  by  Pmd  the  probability  of  declaring  that  the  target  is  in  foliage,  assuming  that  the 
target  is  in  S.  We  denote  by  Pfa  the  probability  of  detecting  the  target  in  S,  assuming  that  the 
target  is  not  in  that  location.  In  the  simulation  setting  it  translates  to  the  following,  as  seen  in  [5]. 
Denote  the  number  of  realizations  of  the  whole  search  scenario  A^/?,  the  number  of  times  the  algorithm 
declared  finding  target  and  the  target  was  not  there  Nfa^  the  number  of  times  target  was  detectable 
(ill  S)  Ndi  and  the  number  of  times  the  target  was  detectable  but  the  algorithm  declared  that  it  is  in 
the  foliage  A/^a/d- 


Pfa  — 


Nfa 


7V/J— ►oo  Nfi 


Pmd  =  lini 


^MD 


Nn^oo  Nq 

During  the  course  of  the  algorithm  the  searcher  moves  in  S,  taking  measurements  with  frequency 
/.  For  easier  description  of  our  decision  making  procedure,  let  us  first  assume  that  the  searcher  moves 
around  S  in  steps.  In  each  step  the  searcher  is  allowed  to  make  several  independent  measurements  with 
his  sensor.  Assume  that  at  each  step  the  searcher  takes  no  independent  measurements,  and  declares 
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detection  of  target  if  at  least  70  +  1  of  the  measurements  are  Is.  The  Ney man- Pearson  criterion  (see 
[4])  allows  us  to  find  no  and  70  that  maximize  the  probability  of  detection  while  the  probability  of 
false  alarm  stays  under  some  prescribed  bound  (Pfa)-  The  Neyman- Pearson  lemma  (sec  [4])  implies 
that  the  optimal  no  and  70  are  the  solutions  to  the  following  optimization  problem. 

Pf^  =  p[fc>7o]+pp[/==7o]=  f; 

i-PMD  =  p[k>io]+pp[k^io]^  £  (2) 

We  first  find  minimal  70  satisfying  (1)  when  p  =  0.  Because  at  this  point  no  is  unknown,  70  =  7o(^o) 
is  a  function  of  no.  Next,  from  the  equation  (1)  we  find  p  =  p{no).  Finally,  we  substitute  70(^0)  and 
p(no)  into  (2)  and  find  the  minimal  no  for  which  the  equation  still  holds.  Taking  no  measurements 
at  each  location  guarantees  that  probability  of  missed  detection  of  the  algorithm  will  be  less  or  equal 
than  P\fo  it  does  not  guarantee  however  that  the  probability  of  false  alarm  of  the  algorithm  is  less 
or  equal  than  Pfa-  Taking  no  measurements  will  be  a  preliminary  criteria  in  our  decision  making 
algorithm:  if  at  least  70  +  1  readings  are  Is  the  searcher  will  assume  that  there  is  a  target  suspect 
at  that  location.  To  achieve  probability  of  false  alarm  less  or  equal  than  Pfa  the  searchers  will  take 
additional  measurements. 

We  denote  by  Tstcyp  the  stopping  time  of  the  algorithm.  The  probability  of  false  alarm  for  one  step 
is  the  probability  of  detecting  the  target  in  S  when  the  target  is  not  in  that  location.  Denoting  the 
total  number  of  steps  taken  by  N,  we  can  express  the  upper  bound  pfa  for  probability  of  false  alarm 
for  each  step  as  follows. 

U-Pfa)^  =  1-PfA  (3) 

From  equation  (3)  we  get 

=  1  -  (1  -  (4) 

Pfa  provides  an  upper  bound  for  the  probability  of  false  alarm  for  each  step  needed  for  the  algorithm 
to  achieve  probability  of  false  alarm  at  most  Pfa-  The  probability  of  missed  detection  for  one  step 
is  the  probability  of  declaring  that  the  target  is  in  foliage  when  the  target  is  S.  Denoting  Pmd  upper 
bound  for  probability  of  missed  detection  for  each  step,  we  get 

4 


24 


A.l.  SPECTRAL  MULTI-SCALE  SEARCH 


Pmd  =  1  -  (1  -  Pmd)  (5) 

Vmd  provides  an  upper  bound  for  the  probability  of  missed  detection  for  each  step.  Using  the  Ncyman- 
Pearson  criterion  again,  we  obtain  the  constants  n\  and  71,  that  will  be  used  by  the  algorithm  in  making 
the  final  decision.  n\  will  be  the  upper  bound  on  the  number  of  measurements  that  the  searcher  may 
take  at  one  step.  From  Neym an* Pearson  lemma  (see  [4])  we  have 

p/a  =  F[fc>7l]+pP[A--  =  71]  =  f:  (6) 

l-Prrui  =  P[k>'n]  +  pP[k  =  ^l]=  (7) 


Summary  of  parameters  and  variables _ 

N  number  of  searchers 

Sd  probability  of  detection  for  a  single  measurement 

Sfa  probability  of  false  alarm  for  a  single  measurement 

Pmd  probability  of  missed  detection  for  one  step 
Pfa  probability  of  false  alarm  for  one  step 

probability  of  missed  detection  of  the  algorithm 
PpA  probability  of  false  alarm  of  the  algorithm 

Tstop  stopping  time  of  the  algorithm 

/  frequency  of  the  sensor  measurements 

Q  probability  for  target  to  be  in  foliage  (0  <  o  <  1) 

6  upper  bound  of  the  radius  of  the  sensor 
A  the  search  area 

F  foliage 

To  find  ni,  we  find  minimal  71  =  7(ni)  satisfying  (6)  when  p  =  0.  Next,  from  the  equation  (7) 
we  find  p  =  p(ni).  Finally,  we  substitute  71  and  p(ni)  into  (7)  and  find  the  minimal  n\  for  which  the 
inequality  still  holds. 

The  decision  making  algorithm 
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Figure  1;  ROC  curves 

1.  At  each  step  the  searcher  takes  no  measurements.  If  the  number  of  Is  is  less  or  equal  than  70, 
the  searcher  decides  that  the  target  is  not  within  the  sensing  area,  and  makes  another  step. 

2.  If  the  number  of  Is  is  greater  than  70,  the  se.archer  starts  making  additional  measurements, 
stopping  after  at  most  ni  measurements.  After  each  additional  measurement  the  searcher  checks 
whether  or  not  the  ratio  of  Is  is  smaller  than  (71  -h  l)/ni.  If  yes,  the  searcher  decides  that  the 
target  is  not  within  the  sensing  area  and  makes  the  next  step.  If  no,  the  searcher  makes  an 
additional  measurement.  If  the  searcher  has  made  ni  measurements  and  the  ratio  of  Is  is 
greater  than  (71  -h  l)/ni,  he  declares  that  the  target  is  detected. 

3.  If  no  detection  occurs  until  time  Tstop,  the  searcher  stops  and  declares  that  the  target  is  in  the 
foliage. 

In  figure  1  we  present  performance  plots  of  our  decision  making  algorithm.  Fixing  constants  no, 
Sd  and  Sfa  we  can  compute  all  corresponding  pairs  Pfa^  Paid  using  equations  (1)  and  (2).  For  fixed 
Sd  and  Sfa  each  color  represents  a  constant  no,  shown  on  the  picture.  The  pairs  Sd  ^  sja  that  wc  use 
are  characteristics  of  real-life  sensors  computed  in  [7]. 

An  estimate  for  the  stopping  time  of  the  algorithm  can  be  obtained  as  follows. 

Tst^  =  1^1  (8) 
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Figure  2:  Search  area  S  with  foliage  (grey),  prior  (red)  and  5000  random  targets 

3  coverage 

In  SMS  search  we  use  design  of  motion  described  in  [3]  for  ?i  searchers  to  achieve  optimal  coverage 
of  the  prescribed  domain.  The  heart  of  the  coverage  method  is  a  Lyapunov-based  control  design 
utilizing  an  Sobolev  space  cost  function.  The  method  in  [3]  allows  to  design  optimal  dynamics 
for  searchers  with  the  goal  to  cover  any  part  of  a  given  area  S.  For  instance,  if  the  location  of  the 
target  is  described  by  a  probability  distribution  P,  the  part  of  S  where  V  >  0  has  to  be  covered.  We 
tested  the  motion  design,  included  in  SMS  search,  for  50  searchers  and  area  S  shown  on  figure  2,  with 
foliage  is  shown  in  grey.  Wc  tested  the  problem  in  case  when  the  prior  probability  distribution  of 
the  target  V  is  uniform  and  in  case  when  the  location  of  the  target  corresponds  to  prior  distribution 
shown  in  red  (higher  probability  corresponds  to  darker  shade  in  figure  2).  As  illustrated  in  figure  3 
and  5,  the  coverage  motion  design  of  SMS  search  guarantees  superior  coverage  according  to  prior 
of  the  target  in  both  uniform  and  non-uniform  cases.  Note  that  searchers  move  with  realistic  second 
order  dynamics.  As  seen  in  figure  5  the  motion  of  the  searchers  depends  on  the  probability  of  the 
target  being  in  a  certain  part  of  the  area:  the  high-probability  regions  are  always  covered  better.  By 
covering  S  according  to  a  probability  distribution  P  coverage  saves  time  by  not  going  to  regions 
where  the  probability  distribution  of  the  target  is  0.  There  is  a  potential  drawback  in  //“^  coverage 
if  V  contains  several  high  peaks.  In  that  case  the  searchers  will  cover  the  regions  close  to  peaks  much 
more  than  needed  to  guarantee  the  desired  precision  threshold  for  SMS  search.  A  clever  way  to  avoid 
over-covering  the  area  is  to  use  \og{V}  instead  of  V.  Figure  6  illustrates  the  reduction  of  peaks  using 
logarithm  by  showing  two  different  cross-sections  of  V  and  log(7^).  Figure  7  shows  the  area  that  has 
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Figure  3:  The  area  S  covered  by  searchers  shown  in  blue  (uniform  prior) 
5000  targets  (3629  targets  not  in  foliage) 


Figure  4:  Typical  coverage  of  log(P)  >  0  by  50  searchers 


Figure  5:  Typical  coverage  of  P  >  0  by  50  searchers 
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Figure  6:  Reduction  of  high  peaks  in  V  (dashed),  by  replacing  it  with  log(P)  (solid  line) 
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Figure  7:  Search  area  S  with  of  \og{V)  >  0  shown  in  red 


Figure  8:  Typical  coverage  of  log(P)  >  0  by  50  searchers 
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to  be  covered  after  taking  logarithm  of  the  prior.  As  illustrated  in  figure  8,  after  reducing  high  peaks 
we  avoid  visiting  certain  areas  too  many  times  and  the  coverage  of  the  desired  parts  of  S  becomes 
more  even. 

4  Spectral  Multi  Scale  Search  (SMS  search) 

Assume  that  we  have  n  searchers  and  an  area  A  where  a  single  target  is  located.  A  may  contain 
foliage  F  which  the  sensors  can  not  penetrate.  If  the  target  is  in  the  foliage  it  is  undetectable  for 
the  searchers.  We  use  a  decision  making  strategy  based  on  the  Neym an- Pearson  lemma,  that  was 
described  in  Section  2,  and  compute  constants  7io>  7o>  and  71  for  given  Ppj\^  ,  Pmd,  sjq  and 
We  use  a  measurement  history  map  to  keep  track  of  target  suspects:  we  divide  the  area  S  into  small 
enough  cells,  and  keep  record  of  sensor  measurements  for  each  cell.  At  the  beginning  of  the  algorithm 
the  belief  map  and  the  list  of  target  suspects  have  no  records.  There  will  be  two  main  modes  for  each 
searcher:  explore  and  recheck.  After  deployment,  all  searchers  start  out  in  explore  mode. 

1.  In  explore  mode  the  searchers  cover  the  search  area  using  coverage  dynamics  and  update 
the  measurement  history  map.  When  the  number  of  measurements  at  a  location  becomes  tiq  we 
check  if  the  number  of  detections  at  the  location  exceeds  70:  if  yes,  the  location  is  added  to  the 
list  of  target  suspects.  Starting  from  the  most  likely  targets  (locations  that  have  the  highest  ratio 
of  positive  measurements),  each  target  suspect  is  assigned  to  an  available  neighboring  searcher 
in  explore  mode.  The  searcher  that  has  been  assigned  a  target  suspect  changes  his  mode  to 
rechecking,  and  moves  to  the  location  of  the  target  on  a  straight  line  with  maximum  speed. 

2.  In  recheck  mode,  the  searcher  has  to  perform  tiq  measurements  flying  above  a  target  suspect  posi¬ 
tion.  After  finishing  the  measurements,  the  searcher  switches  to  explore  mode.  After  rechecking, 
if  the  ratio  of  detections  to  the  number  of  measurements  exceeds  we  keep  the  location  in  the 
list  of  target  suspects,  otherwise  we  remove  it  from  the  list. 

3.  When  the  number  of  measurements  at  a  location  becomes  ni,  we  check  if  the  number  of  detections 
at  the  location  exceeds  71:  if  yes,  we  declare  that  the  target  is  found  and  stop  the  search. 

4.  If  the  algorithm  reaches  stopping  time  T^top,  without  declaring  a  detection,  the  algorithm  declares 
that  the  target  is  in  the  foliage. 

We  tested  the  SMS  search  algorithm  for  50  searchers  on  a  rectangular  area  A  shown  in  figure  2. 
Foliage  is  shown  in  dark  grey  and  the  prior  distribution  of  the  target  is  shown  in  red.  Each  searcher 
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Search  bme  (sec)  Search  bfDe  (sec) 


Figure  9:  Histograms  of  SMS  search  for  non-uniform  prior  and  log-non-uniform  prior 


Search  time  (sec) 


Figure  10:  Histogram  of  SMS  search  for  uniform  prior 
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Figure  11:  Proof  that  the  algorithm  stays  within  the  desired  probability  thresholds 

has  a  fixed  uncertain  sensor  range  that  changes  periodically  with  a  fixed  period.  The  probabilities  of 
detection  and  false  alarm  for  a  single  sensor  are  Sd  =  0.8  and  5/a  =  0.2  respectively.  The  goal  is  to 
minimize  search  time  while  satisfying  requirements  Po.group  >  0.9  and  PpA^group  <  0.1.  In  figure  11 
we  show  that  Pp, group  fi^nd  Pfa, group  converges  above  and  below  the  required  limits,  respectively,  as 
the  number  of  realizations  of  SMS  search  increases.  We  tested  the  SMS  algorithm  in  5000  experiments 
with  randomly  generated  targets  for  uniform  prior,  shown  in  figure  4,  and  in  case  of  non-uniform  prior, 
shown  in  figure  2.  In  figure  9  we  compare  histograms  of  5000  experiments  of  SMS  search  performed 
on  S  with  non-uniform  prior  shown  in  figure  2  and  with  log- non-uni  form  prior  shown  in  figure  7.  The 
search  and  detection  time  statistics,  also  presented  in  figure  9,  show  that  the  median  search  time  of 
SMS  applied  to  non-uniform  prior  is  15%  bigger  than  in  case  of  log-non-uniform  prior  and  the  median 
absolute  deviation  of  SMS  detection  time  applied  to  non-uniform  prior  is  40%  bigger  than  in  case  of 
log-non-uniform  prior.  The  histogram  of  SMS  search  performed  on  S  with  uniform  prior  is  in  figure  10. 
In  figure  12  the  median  of  the  search  time  is  shown  as  a  function  of  the  number  of  realizations  of  SMS 
search.  Using  coverage  on  non-uniform  prior  (see  figures  2,  5)  results  in  a  median  search  time  as 
compared  with  coverage  for  uniform  prior  (see  figure  3).  Taking  logarithm  of  the  non-uniform 
prior  (see  figures  8,  7,  6)  helps  to  reduce  the  median  search  time  even  further. 

The  resulting  median  detection  time  was  203  sec,  median  search  time  was  400  sec,  median  absolute 
deviation  (MAD)  was  229  sec. 
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Figure  12:  Median  search  time  of  SMS  algorithm 
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5  Conclusions 

Ill  this  paper  we  explore  performance  of  search  algorithms  in  presence  of  uncertainty  in  sensor  radius. 
The  introduced  SMS  search  algorithm  demonstrates  excellent  performance  in  presence  of  uncertainty 
as  shown  in  the  table  below.  We  designed  the  SMS  algorithm  maximizing  its  effective  use  of  assets: 
the  search  time  is  inversely  proportional  to  the  number  of  searchers. 


Comparison  of  SMS  search  under  different  conditions 


Algorithm 

Median  Detection  Time 

Median  Search  Time 

Median  Absolute  Deviation 

SMS  (no  uncertainty) 

169  sec 

303  sec 

226  sec 

SMS 

203  sec 

400  see 

229  sec 

Billiard  search 

587  sec 

1190  sec 

534  see 

In  the  above  table  we  compare  statistics  for  SMS  search  without  uncertainty,  SMS  search  with 
periodically  changing  uncertain  sensor  radius,  and  Billiard  search  when  searchers  start  out  in  ran¬ 
dom  directions  and  move  in  straight  lines,  reflecting  when  they  reach  the  border.  We  ran  computer 
simulations  of  SMS  search  conducting  5000  independent  experiments  for  each  scenario.  Computer 
simulations  show  that  Median  Detection  Time,  Median  Search  Time  and  Median  Absolute  Deviation 
of  SMS  search  in  presence  of  uncertainty  are  very  close  to  the  corresponding  SMS  search  data  without 
uncertainty  in  sensor  radius. 
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The  median  absolute  deviation  of  SMS  search  time  is  1.5  times  smaller  than  that  of  Billiard  search; 
median  search  time  of  SMS  search  is  1.6  times  smaller  than  that  of  Billiard  search;  median  detection 
time  of  SMS  search  1.7  times  smaller  than  that  of  Billiard  search. 
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1  Overview 

Achieving  the  Surveillance  Milestone  requires  the  design  of  trajectories  for  aerial  vehicles  with 
uncertain  sensors  moving  over  a  large  complex  terrain  so  that  the  time  to  locate  a  stationary  target 
on  the  ground  is  significantly  less  than  the  time  required  by  straightforward  lawnmower  trajectories. 
Because  the  individual  sensor  measurements  include  non-zero  probabilities  for  both  missed  detection 
and  false  alarm,  and  the  sensor  footprint  size  is  uncertain;  the  final  statement  provided  by  the  search 
algorithm  as  to  the  location  of  the  target  cannot  be  provided  with  absolute  certainty.  Rather,  the 
final  declaration  of  the  target  location  must  meet  confidence  thresholds  specified  a  priori  for  the 
probability  that  the  stated  target  location  is  correct.  In  this  report,  we  present  the  Dynamic 
Greedy  Search  (DyGS)  algorithm  for  dynamically  generating  vehicle  trajectories  that  achieves 
this  Surveillance  Milestone  while  ensuring  that  the  method  is  robust  to  sensor  failure,  and  is  easily 
implemented  for  vehicles  with  complex  dynamics  and  for  regions  with  arbitrary  foliage  distributions. 
The  DyGS  algorithm  achieves  a  1.9  times  speed-up  over  lawnmower  methods. 

Two  essential  components  of  the  DyGS  search  strategy  presented  here  are  a  dynamic  trajectory 
generation  algorithm  and  a  precise  sensor  decision  algorithm. 

The  trajectory  generation  algorithm  provides  dynamically  updated  trajectories  that  seek  to 
maximize  the  probability  of  finding  the  target  while  ensuring  consistency  with  the  modeled  dynam¬ 
ics  and  limited  control  authority  of  the  vehicles.  A  hallmark  of  the  DyGS  search  strategy  is  that 
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it  requires  no  modification  to  handle  vehicles  with  under-actuated  or  complex  dynamics.  Further¬ 
more,  the  algorithm  can  be  applied  to  real  vehicles  in  which  an  accurate  model  of  the  dynamics  is 
not  known,  and  only  recorded  maneuvers  of  the  real  flight  dynamics  are  provided. 

The  sensor  decision  algorithm  takes  as  its  input  the  history  of  measurements  obtained  during 
the  search  up  to  the  current  time.  Given  these  raw'  inputs,  the  decision  algorithm  must  decide  when 
the  search  may  be  terminated  and  a  declaration  of  the  target  location  can  be  provided  that  satisfies 
the  specified  thresholds  of  ceorrectness.  A  classical  approach  to  dealing  with  measurement  data  is 
to  sample  the  incoming  measurements  onto  a  spatial  grid,  and  then  to  calculate  the  probability  for 
the  target  to  be  located  at  each  cell  in  the  grid.  Here,  we  use  a  grid-free  approach  that  instead  stores 
the  measurements  as  raw  atomistic  data  so  that  inferences  of  the  target  location  are  made  using 
the  raw  measurements  in  their  most  precise  form.  This  approach  has  several  advantages  in  that  it 
removes  approximation  errors  incurred  by  gridding,  maximizes  the  use  of  information  received  in 
an  information-theoretic  sense,  and  also  forgoes  the  need  to  impose  a  grid  and  a  consequent  scaling 
on  the  search  domain  that  is  not  intrinsic  to  the  problem.  In  practice,  imposition  of  a  grid  can  lead 
to  ambiguity  and  ill-conditioned  interpretation  of  the  probabilities  in  each  cell. 

2  The  Surveillance  Milestone 

VVe  now  provide  a  precise  description  of  the  Surveillance  Milestone  problem.  In  order  to  assess  the 
efRciency  of  various  search  algorithms  we  have  developed  a  specific  test  problem  against  which  all 
the  methods  may  be  benchmarked.  How'ever,  an  important  point  is  that  the  DyGS  search  algo¬ 
rithm  is  generally  and  easily  applicable  to  arbitrary  search  domains  and  foliage  distributions,  and 
the  specific  domain  used  here  is  simply  an  example  case.  In  particular,  the  search  domain  need  not 
be  a  simply  connected  region. 

The  Search  Domain: 

1.  The  search  domain,  Z),  is  a  golden  rectangle  with  an  area  of  1  square  kilometer. 

2.  Regions  of  the  search  domain  are  designated  as  foliage  as  indicated  in  Figure  1. 

3.  A  probability  density  /  with  support  on  D  provides  the  probability  density  for  finding  the 
target  at  each  location  in  the  search  domain.  Since  there  is  exactly  one  target  to  be  found, 
Jp  /  dA  —  1.  Target  locations  are  sampled  from  this  probability  density  function. 

The  Vehicle  Model: 

1.  Fifty  vehicles,  each  equipped  with  a  single  sensor,  are  available  to  search  the  domain. 

2.  The  vehicles  are  initially  deployed  at  rest  on  a  5  by  10  grid  with  5m  spacing  in  the  lower  left 
corner  of  the  domain. 

3.  The  dynamics  of  the  search  vehicles  are  modeled  as  a  simple  double  integrator: 

x  =  u  (1) 

where  x  is  the  location  of  the  vehicle  in  the  plane,  and  u  is  a  control  force  that  must  be  chosen 
subject  to  the  constraints  that  the  maximum  allowed  magnitude  of  the  velocity  vector  is  10 
m/s,  and  the  maximum  allowed  magnitude  of  the  acceleration  vector  is  5  m/s^. 
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Figure  1:  The  search  domain  shown  here  is  used  to  benchmark  search  algorithms  in  the  Surveillance 
Milestone.  Green  areas  represent  foliage  in  which  the  target  can  hide  without  being  seen  by  the  sensors. 


The  Sensor  Model: 

1.  Each  sensor  takes  a  measurement  every  0.5  seconds. 

2.  During  a  measurement,  the  area  scanned  by  the  sensor  is  circular  and  lies  directly  below  the 
vehicle.  The  radius  of  the  area  scanned  by  the  sensor  changes  every  50  seconds  and  is  chosen 
uniformly  from  the  interval  5m  to  lOin. 

3.  If  the  target  is  outside  the  foliage  and  inside  the  small  circular  area  scanned  by  the  sensor, 
then  the  sensor  reports  with  probability  0.8  that  the  target  has  been  seen.  In  other  words, 
the  sensor  probability  of  detection,  p^,  is  0.8. 

4.  If  the  scanned  area  includes  regions  without  foliage  and  the  target  is  not  in  these  open  regions, 
then  with  probability  0.2  the  sensor  reports  seeing  the  target,  i.e.  the  sensor  probability  of 
false  alarm,  pfa,  is  0.2. 

5.  For  all  other  cases,  the  sensor  reports  that  the  target  was  not  seen. 

6.  When  the  sensor  reports  seeing  the  target,  the  exact  location  of  the  target  is  provided.  If 
the  sensor  provides  a  false  alarm,  a  location  inside  the  open  region  inside  the  scanned  area 
is  randomly  generated  and  returned.  This  location  is  stored  by  the  sensor  measurement 
generator  and  returned  during  future  false  alarms  for  cases  when  the  scanned  area  includes 
this  location. 

7.  With  each  measurement,  the  sensor  also  provides  the  exact  location  of  the  vehicle. 

Information  Available  to  the  the  Search  Algorithm: 

1.  The  search  algorithm  is  provided  with  the  exact  geometry  of  the  search  domain  and  foliage, 
the  probability  distribution  of  the  target  locations,  and  the  lower  and  upper  bounds  for  the 
radius  of  the  circular  area  scanned  by  the  sensor. 

2.  At  each  timestep  (every  0.5  seconds),  the  algorithm  is  provided  with  measurements  from  each 
of  the  sensors.  The  measurement  data  includes  the  precise  location  of  the  sensor,  and  one  of 
the  following  statements: 

(a)  No  target  was  seen. 
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(b)  The  target  was  seen  at  the  location  (xm,2/m)- 
Information  Provided  by  the  Search  Algorithm: 

1.  At  each  time  step,  the  search  algorithm  must  provide  trajectories  along  which  the  sensors 
must  move.  The  trajectories  must  be  consistent  with  the  specified  vehicle  dynamics. 

2.  The  algorithm  must  terminate  the  search  and  provide  a  final  declaration  of  the  target  location. 
This  declaration  must  be  one  of  the  following: 

(a)  The  target  is  in  the  foliage, 

(b)  The  target  is  in  the  open  area  and  is  located  at  the  position  y^). 

Benchmarking  Search  Algorithms: 

1.  NT  represents  the  number  of  target  locations  that  are  sampled  from  the  target  probability 
density.  We  choose  NT  =  5000. 

2.  For  each  target  location,  the  search  algorithm  is  run  until  the  search  is  terminated  and  a 
declaration  of  the  target  location  is  made.  The  time  taken  to  perform  the  search  is  recorded. 

3.  After  all  NT  searches  have  been  conducted,  the  correctness  of  the  declarations  is  checked. 
Let 

ND  :  = 

NFA  := 

NMD  := 

Then  compute 

Ffa  •=  7^  (Algorithm  f)robabilty  of  false  alarm), 

:=  (Algorithm  probabilty  of  missed  detection). 

The  algorithm  is  considered  sufficiently  correct  if  Pmo  and  Pfa  are  both  less  than  0.1. 

4.  If  the  algorithm  is  sufficiently  correct,  then  the  median  search  time  of  all  the  NT  search  times 
is  computed. 

5.  Sufficiently  correct  search  algorithms  are  ranked  by  comparing  their  median  search  times. 

The  Surveillance  Milestone  Problem  Statement: 

Design  a  search  algorithm  so  that  the  median  search  time  computed  using  the  methodology  de¬ 
scribed  above  is  less  than  the  median  search  time  computed  for  a  search  algorithm  that  uses 
systematic  lawnmow'er  type  trajectories. 


(Number  of  detectable  targets)  The  number  of  target  locations  in  the 
open  region, 

(Number  of  false  alarms)  The  number  of  declarations  stating  that  the 
target  was  at  a  specific  point  in  the  open  region  that  were  not  correct, 
(Number  of  missed  detections)  The  number  of  declarations  stating  that 
the  target  was  in  the  foliage  when  the  target  was  in  fact  in  the  open 
region. 
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3  The  Dynamic  Greedy  Search  Algorithm 

As  mentioned  in  the  overview,  the  Dynamic  Greedy  Search  (DyGS)  strateg>^  comprises  two  al¬ 
gorithms:  a  grid-free  decision  algorithm  for  storing  measurement  data  and  deciding  when  a  final 
declaration  as  to  the  target  location  can  be  made,  and  a  trajectory  generation  algorithm  for  dy¬ 
namically  generating  trajectories  that  are  consistent  with  the  vehicle  dynamics  and  maximize  the 
probability  of  finding  the  target. 

3.1  The  Grid-Free  Decision  Algorithm 

The  grid-free  approach  provides  a  method  for  storing  and  interpreting  the  measurement  data  in 
a  manner  that  is  computationally  efficient  while  maximizing  the  utility  of  the  raw  measurements. 
The  central  data  structure  maintained  by  this  algorithm  is  a  list  of  suspect  locations.  Whenever 
a  sensor  reports  that  the  target  has  been  seen  at  a  particular  location,  and  this  location  has  not 
previously  been  identified,  this  new  location  is  added  to  the  list  of  suspect  locations.  Associated 
with  each  suspect  location  in  the  list  is  a  repository  for  all  measurements  that  have  ever  been  taken 
at  locations  that  are  less  than  the  maximum  possible  sensor  radius  away  from  the  suspect  target 
location.  Hence,  along  with  the  suspect  location,  we  store  every  measurement  that  could  possibly 
have  included  this  location  in  its  scanning  area.  With  all  the  locally  relevant  measurements  in 
hand,  we  can  conveniently  compute  the  local  probability  that  the  target  is  located  at  each  suspect 
location  in  the  list.  For  the  case  when  the  algorithm  is  told  a  priori  that  there  is  only  one  target,  all 
measurements  are  globally  dependent  -  a  null  measurement  in  one  location  increases  the  probability 
for  positive  measurements  elsewhere  in  the  domain.  For  this  case,  it  is  conceivable  that  we  could 
all  the  global  measurement  data  to  compute  the  probability  that  the  target  is  located  at  each  suspect 
location  exactly;  however,  this  procedure  would  require  computationally  expensive  integrations  and 
summations  to  compute  the  conditional  probabilities.  The  approach  presented  here  computes  only 
a  local  probability  but  in  a  manner  that  uses  the  data  precisely.  Furthermore,  the  probabilities 
become  precise  for  the  case  when  the  number  of  targets  is  not  known  and  measurements  separated 
by  more  than  twice  the  maximum  sensor  radius  are  independent. 

The  probability  that  the  target  is  located  at  a  suspect  location  is  updated  whenever  a  local 
measurement  is  received  using  a  Bayesian  update  scheme.  Let  denote  the  current  set  of 

NS  suspect  locations.  Let  denote  all  the  NM  measurements  local  to  sj  that  have  been 

received  up  to  the  current  time.  Finally,  let  Tj  denote  the  event  that  the  target  is  in  fact  located  at 
Sj.  When  a  new  suspect  location  is  identified  and  added  to  the  suspect  list,  the  probability  that  the 
target  is  at  this  location  is  initialized  to  the  value  of  the  probability  density  function  for  the  target 
distribution  at  this  point.  Then,  the  probability  is  updated  using  all  measurements  previously 
recorded.  Since  we  are  using  the  local  approximation  (that  distant  measurements  are  independent) 
we  need  only  use  nearby  measurements  to  perform  this  calculation.  As  the  search  proceeds  and 
more  local  measurements  are  received,  the  probability  is  further  updated.  The  formula  for  updating 
the  probability  associated  with  suspect  location  sj  given  a  new  measurement  is: 
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P(  Tj  I  m]  )  := 


P(mi  \Tj)^f(sj) 


PK+i  I  •  fi^j)  +  I  ~  'Tj  )  •  (1  -  f{sj)) 


k  =  0, 


P(  Tj  I  rni^,  )  := 


I  Tj  )  •  P(  Tj  I  mi) 


P(mi^,  I  Ti  )  •  P(  T,  I  mi)  +  P(mi^,  |  -  T,  ) .  P(  -  Tj  \  mi) 


Remark  1: 

The  uncertainties  introduced  by  the  sensor  probabilities  for  false  alarm  and  missed  detections,  as 
well  as  the  uncertainty  with  respect  to  the  sensor  radius  are  accounted  for  in  the  right  hand  side 
in  the  following  way: 

If  the  measurement  did  not  observe  the  target,  then 

P(  ^i+i  \  Tj)  =  {I  —  pd)  '  P (vehicle  was  in  range  of  Sj)  -f  (1  —  Pfa)  *  P( vehicle  was  out  of  range  of  Sj). 
If  the  measurement  did  observe  the  target,  then 

P(  mi_j_j  I  Tj  )  =  pd  '  P(vehicle  was  in  range  of  Sj)  -f  pfa  •  P(vehicle  was  out  of  range  of  sj). 

The  probability  that  the  vehicle  was  in  or  out  of  range  of  the  suspect  location  is  determined 
by  the  specified  uncertainty  in  the  sensor  radius.  For  the  specified  range  of  values  for  the  sensor 
radius,  and  denoting  the  vehicle  position  during  measurement  m^  as  we  have  :=  ||x^  “  5j||, 

f  1  if  <  5 

P(vehicle  was  in  range  of  Sj)  =  <  2  —  0.2?^  if  5  <  <  10  ,  (2) 

[  0  if  >  10 

and 

P( vehicle  was  out  of  range  of  Sj)  =  1  —  P (vehicle  was  in  range  of  sj) .  (3) 

Making  the  assumption  that  measurements  separated  by  more  than  twice  the  maximum  sensor 
radius  are  uncorrelated  (which  is  equivalent  to  assuming  the  number  of  targets  is  unknown)  implies 
that  we  only  need  to  consider  measurements  that  are  nearby  the  suspect  location  when  performing 
the  probability  update. 

Remark  2: 

Notice  that  a  separate  probability  space  is  assigned  locally  to  each  suspect  location,  and  that  each 
probability  space  is  partitioned  into  only  two  possible  events:  the  target  is  at  sj  (denoted  Tj)  or 
the  target  is  not  at  sj  (denoted  ~  Tj).  Since  all  the  probabilities  are  computed  locally,  there  is 
no  requirement  that  the  sum  of  all  the  probabilities  at  all  the  suspect  locations  is  unity.  In  other 
words,  we  should  not  expect  \  ^{)  =  1-  What  is  guaranteed  by  construction  is  that 

P(  Tj  I  mi  )  +  P(  ~  Tj  I  mi  )  =  1 . 
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These  local  probabilities  provide  a  sense  for  the  most  likely  target  locations  based  on  local 
measurements  only. 


At  every  timestep,  the  data  structure  described  above  provides  a  ranked  list  of  the  most  likely 
locations  where  the  target  is  hiding  given  the  measurement  history.  What  remains,  is  for  the  al¬ 
gorithm  to  determine  when  a  target  is  located  at  a  specific  suspect  location  with  sufficiently  high 
probability  that  the  search  can  be  terminated  and  a  declaration  of  the  target  location  can  be  made. 

The  algorithm  must  also  be  able  to  determine  with  sufficient  confidence  of  correctness  that  the 
target  is  in  the  foliage.  To  effect  these  actions,  we  introduce  three  parameters: 
MinimumNumberOfMeasurementsToCheck,  MinimumNumberOfMeasurementsToDeclare,  and  Maximum- 
SearchTime  that  we  will  now  proceed  to  describe. 

Suspect  Checking: 

The  default  behavior  of  the  search  vehicles  is  to  browse  the  search  domain  and  to  gather  measure¬ 
ments  in  regions  where  the  target  is  most  likely  to  be  found.  (More  on  how  these  search  trajectories 
are  generated  is  provided  in  Section  3.2).  As  the  vehicles  browse  the  domain,  a  list  of  suspect  lo¬ 
cations  and  the  local  probability  that  the  target  is  in  fact  at  these  locations  is  generated.  If  the 
local  probability  that  the  target  is  located  at  a  particular  suspect  location  rises  above  0.5,  and  the 
number  of  measurements  used  to  compute  this  probability  is  greater  than  or  equal  to  Minimum- 
NumberOfMeasurementsToCheck,  then  the  algorithm  will  command  the  most  nearby  search  vehicle 
to  fly  directly  to  the  suspect  location  to  gather  more  measurements  to  check  the  status  of  the  sus¬ 
pect.  During  this  checking  process,  a  count  of  the  number  of  certain  measurements  (measurements 
obtained  while  the  sensor  is  less  than  5m  away  from  the  suspect  location)  is  kept.  This  requirement 
effectively  removes  uncertainty  due  to  the  variable  sensor  radius.  When  the  number  of  certain  mea¬ 
surements  reaches  MinimumNumberOfMeasurementsToDeclare,  the  algorithm  has  sufficient  data  to 
determine  the  status  of  the  suspect  location.  If  more  than  half  of  the  certain  measurements  indicate 
that  the  target  is  present,  then  the  algorithm  can  terminate  the  search  and  declare  that  the  target 
is  located  at  the  suspect  location.  Otherwise,  the  suspect  location  is  flagged  as  checked,  the  search 
vehicle  resumes  browsing,  and  the  search  continues. 

Search  Termination: 

The  search  algorithm  proceeds  with  sensors  simultaneously  browsing  and  checking  suspect  loca¬ 
tions.  The  longer  the  search  continues  without  uncovering  the  location  of  the  target,  the  greater 
the  probability  that  the  target  is  located  in  the  foliage.  When  the  search  time  reaches  Maximum- 
SearchTime,  the  search  is  terminated  and  a  declaration  is  made  that  the  target  is  in  the  foliage. 
Choosing  larger  values  of  Maxim umSearchTime  will  provide  greater  confidence  in  the  correctness  of 
this  declaration. 

Choosing  Values  for  the  Parameters: 

In  short,  the  values  of  MinimumNumberOfMeasurementsToCheck,  MinimumNumberOfMeasurementsToDe- 
clare,  and  MaximumSearchTime  are  chosen  so  as  to  minimize  the  median  search  time.  In  this  way, 
freedom  in  the  choice  of  the  parameter  values  allows  the  search  algorithm  to  be  optimized  for  the 
specific  search  problem  at  hand. 

In  practice,  MinimumNumberOfMeasurementsToCheck  is  chosen  from  the  nominal  set  of  values 
{2,3, 4, 5},  and  then  a  simple  search  is  performed  to  find  the  values  of  the  remaining  two  parameters 

7 


43 


A.2.  DESIGN  OF  SEARCH  TRAJECTORIES  FOR  VEHICLES  WITH  UNCERTAIN 
SENSORS 


so  that  the  search  time  is  minimized. 

It  should  be  noted  that  for  a  given  sensor  pd  and  pfa,  the  Neym an- Pearson  Lemma  provides 
a  minimum  number  of  measurements  that  are  required  to  test  the  hypothesis  that  the  suspect  is 
indeed  the  target,  and  the  number  of  positive  measurements  that  are  required  in  order  to  declare 
that  the  suspect  is  indeed  the  target  with  sufficient  correctness.  These  theoretical  results  are 
helpful  in  guiding  the  choice  of  the  parameters,  however,  in  practice  these  choices  tend  to  be  too 
conservative.  The  freedom  to  choose  and  adjust  the  parameters  allows  the  algorithm  to  exploit 
structure  in  the  given  problem.  Inhomogeneous  structure  arises,  for  example,  from  the  uneven 
distribution  of  foliage,  the  shape  of  the  target  probability  distribution,  the  geometry  of  the  domain, 
and  the  concentrated  deployment  location  of  the  vehicles.  All  these  factors  introduce  non-trivial 
effects  that  are  not  accounted  for  in  a  straightforward  application  of  the  Neym  an- Pearson  lemma 
that  assumes  a  homogeneous  structure  and  distribution  of  events. 

Finding  optimal  values  of  the  remaining  two  parameters,  MinimumNumberOfMeasurementsToDe- 
clare  and  Maxima mSearchTime,  proceeds  very  quickly  because  of  the  relatively  simple  functional 
dependence  of  the  search  time  on  the  parameters.  VVe  choose  the  parameters  as  low  as  possible  to 
decrease  the  search  time,  subject  to  the  following  constraints: 

Cl.  Decreasing  the  stopping  time  increases  P\id  (The  algorithm  is  too  hasty  to  declare  that  the 
target  is  in  the  foliage). 

C2.  Decreasing  MinimumNumMeasurements  increases  PpA  (The  algorithm  is  too  hasty  to  declare 
that  the  target  is  located  at  a  suspect  location). 

In  practice,  we  sequentially  decrease  these  two  parameters  as  much  as  possible  without  violating 
the  specified  /Va  and  Pmd-  This  is  a  simple  bisection  search  that  requires  perhaps  5  to  10  runs 
on  each  parameter.  Each  run  (which  includes  5000  searches)  takes  2  to  4  minutes  on  a  cluster  at 
Caltech. 

The  main  features  of  the  sensor  decision  algorithm  are  summarized  as  follows: 

1.  No  grid  is  required.  The  original  problem  has  no  inherent  grid  and  we  do  not  impose  one. 

2.  We  do  not  need  to  assign  measurements  to  a  grid.  This  process  inherently  introduces  uncer¬ 
tainty  that  is  not  a  part  of  the  original  problem. 

3.  The  final  declaration  of  the  target  position  is  exact.  It  is  either  at  that  exact  location  or  it  is 
not. 

4.  An  optimized  Neym  an- Pearson  criterion  is  used  in  the  local  binary  hypothesis. 

5.  Time  is  optimized  while  ensuring  the  constraints  on  P^a  Pmd  satisfied. 

3.2  Dynamic  Greedy  Trajectory  Generation 

We  now  describe  the  algorithm  for  dynamically  generating  trajectories  that  are  consistent  with 
the  vehicle  dynamics.  In  essence,  the  trajectories  for  each  vehicle  are  chosen  from  pre-computed 
trajectory  segments  so  as  to  maximize  passage  through  regions  of  the  domain  in  which  the  target 
is  most  likely  to  be  found  and  that  have  not  previously  been  visited. 
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Figure  2:  The  trajectory  library  consists  of  motion  primitives  that  can  be  pieced  together  to  generate  a 
vehicle  trajectory  that  is  consistent  with  the  dynamics  and  limited  control  authority  of  the  vehicles.  Elements 
in  the  trajectory  library  are  shown  in  figures  (a)  through  (e)  in  order  to  emphasize  how  the  library  allows 
for  a  range  of  motion  that  covers  the  multiple  scales  of  the  search  domain. 


Library  of  Trajectories: 

Each  sensor  has  a  library  of  trajectories  stored  in  memory.  A  trajectory  consists  of  a  list  of 
waypoints  where  each  waypoint  is  a  triple  {x,y,0)  describing  the  position  and  heading  angle  of 
the  vehicle  along  the  trajectory  at  time  intervals  commensurate  with  the  frame  rate  of  the  camera. 
The  library  is  pre-coin pu ted  and  can  be  stored  in  memory  at  the  factory  where  the  vehicles  are 
manufactured. 

The  library  currently  implemented  in  the  DyGS  algorithm  has  205  trajectories.  As  shown  in 
Figure  2,  the  library  includes  trajectories  ranging  from  sharp  left  to  sharp  right  turns  and  range  in 
length  from  12  seconds  to  120  seconds.  This  library  can  be  easily  enriched  with  more  trajectories  or 
replaced  with  new  trajectories  without  requiring  any  changes  to  the  search  algorithm.  A  guideline 
for  choosing  trajectories  to  include  in  the  library,  is  that  they  should  cover  all  scales  of  the  search 
domain. 

Vehicle  Constraints: 

By  construction,  the  trajectories  in  the  library  satisfy  the  dynamic  constraints  of  the  vehicles.  For 
the  case  when  the  dynamics  is  modeled  with  a  double  integrator,  the  trajectories  are  designed  so 
that  the  vehicles  always  move  at  the  maximum  allowed  velocity,  and  accelerations  are  only  applied 
perpendicular  to  the  direction  of  motion. 

Choosing  Trajectories  from  the  Library: 


9 


45 


A.2.  DESIGN  OF  SEARCH  TRAJECTORIES  FOR  VEHICLES  WITH  UNCERTAIN 
SENSORS 


At  each  time  step,  each  sensor  is  moved  to  the  next  waypoint  along  its  current  trajectory.  If  the 
sensor  is  at  the  end  of  its  current  trajectory,  then  the  sensor  must  be  assigned  a  new  trajectory 
from  the  library.  The  sensor  performs  a  scan  through  the  trajectories  in  the  library  to  determine 
which  trajectory  will  maximize  the  time-averaged  probability  of  the  prior  visited  during  the  trajec¬ 
tory.  Visits  to  locations  in  the  foliage  and  outside  the  search  domain  are  considered  to  have  zero 
probability.  Also,  any  location  along  the  trajectory  that  has  been  previously  visited  more  than 
MaxNumVisits  times,  is  considered  to  have  zero  probability.  MaxNum Visits  is  a  parameter  that  is 
chosen  to  adjust  the  ‘‘thickness”  of  the  coverage.^ 

Computational  Effort: 

The  computational  overhead  required  to  implement  this  trajectory  generation  scheme  is  minimal 
since  at  each  time  step,  the  majority  of  vsensors  are  simply  moved  to  the  next  waypoint  along  their 
current  trajectory.  Computation  is  only  required  when  a  sensor  reaches  the  end  of  its  assigned  tra¬ 
jectory.  A  search  conducted  by  fifty  sensors  can  be  simulated  100  times  faster  than  real  time  on  a 
single  modest  CPU.  The  computational  effort  of  the  search  is  dominated  by  the  decision  algorithm 
(updating  suspect  lists  etc.)  and  not  the  trajectory  optimization. 

Extensions: 

The  dynamic  trajectory  generation  algorithm  can  be  easily  extended  to  more  complicated  dynamics 
by  simply  replacing  the  library;  no  changes  to  the  code  are  necessary.  For  example,  we  have  success¬ 
fully  implemented  search  with  vehicles  whose  dynamics  is  described  by  a  simple  three-dimensional 
model  for  under- actuated  helicopter  dynamics.  The  helicopters  are  modeled  as  rigid  bodies  with 
controls  for  forward  and  roll  pitch  of  the  main  blades,  as  well  as  a  yaw  control  from  the  tail  bleides. 
Conceivably,  the  trajectory  generation  algorithm  can  be  applied  to  a  vehicle  in  which  an  accurate 
model  of  the  dynamics  is  not  known;  the  trajectory  library  can  be  generated  by  recording  the 
motion  of  the  vehicle  as  it  is  driven  through  various  maneuvers  by  an  actual  pilot. 

Robustness: 

Since  the  trajectories  are  dynamically  generated,  they  easily  and  quickly  adapt  to  sudden  changes 
or  updates  in  the  prescribed  search  domain,  foliage  distribution,  or  initial  target  probability  distri¬ 
bution.  In  contrast,  the  trajectories  used  in  a  systematic  search  strategy  such  as  the  lawnmower 
approach,  are  pre-corn pu ted  and  difficult  to  alter  dynamically  if  new  information  about  the  search 
domain  is  provided  or  it  becomes  apparent  that  one  of  the  sensors  has  failed.  Moreover,  lawn- 
mower  styled  trajectories  become  increasingly  difficult  to  design  for  search  domains  with  complex 
geometry  and  foliage  distributions.  The  ability  of  the  DyGS  algorithm  to  dynamically  update  the 
trajectories  and  the  resulting  chaotic  nature  of  the  these  trajectories  affords  them  more  robustness 
when  faced  with  search  domains  that  have  complex  geometries. 

4  Results  and  Conclusion 

A  search  method  that  uses  systematic  lawnmower  styled  trajectories  was  implemented  for  purposes 
of  comparison  with  the  DyGS  algorithm.  When  applied  to  the  test  domain  described  in  Section  2, 

*  MaxNumVisits  can  be  adjusted  continuously  through  the  positive  reals  via  use  of  a  coin.  For  example,  to  realize 
MaxNumVisits  =  3.67  we  ask  if  a  random  number  in  the  unit  interval  is  greater  than  0.67.  If  yes,  then  choose 
MaxNumVisits  =  3,  otherwise  choose  MaxNumVisits=  4. 


10 


46 


A.2.  DESIGN  OF  SEARCH  TRAJECTORIES  FOR  VEHICLES  WITH  UNCERTAIN 
SENSORS 


the  lawnmower  method  achieves  a  median  search  time  of  887.5  seconds.  The  median  search  time 
obtained  for  the  DyGS  algorithm  is  458.5  seconds.  We  conclude  that  the  DyGS  algorithm  achieves 
the  Surveillance  Milestone  by  producing  a  1.9  times  speed  up  over  the  systematic  lawnmower  search 
strategy.  What  is  more,  the  DyGS  algorithm  provides  robustness  to  sensor  failure  and  to  changes 
in  the  search  domain  geometry  and  foliage  distribution.  The  DyGS  algorithm  is  computationally 
fast,  and  can  handle  vehicles  with  complex,  under- actuated,  or  even  unknown  vehicle  dynamics. 


11 


47 


A.3.  COLLISION  AVOIDANCE  AND  SURVEILLANCE  WITH  AUTONOMOUS  VEHICLES 


A.3 


Collision  avoidance  and  surveillance  with  autonomous  vehi¬ 
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Abstract 

The  main  aim  of  this  paper  is  two  fold.  First  is  to  present  a  new  technique 
for  collision  avoidance  of  mechanical  systems  and  second  is  to  demonstrate  the 
role  of  collision  avoidance  to  enhance  surveillance.  The  traditional  techniques 
for  collision  avoidance  arc  based  on  shaping  the  potential  energy  [18]  or  by 
introducing  gyroscopic  forces  into  the  system  [6].  In  this  paper,  we  close  the 
story  by  introducing  kinetic  shaping  for  collision  avoidance  in  the  spirit  of  the 
Method  of  Controlled  Lagrangians  [5,  4).  We  also  provide  analytic  guarantee 
for  no  collision  under  some  mild  conditions  depending  only  upon  the  energy  and 
momentum  of  the  system  as  it  enters  into  a  collision  avoidance  mode.  The  cor¬ 
responding  control  effort  is  compared  with  known  techniques.  For  an  example 
vehicle  model,  it  is  shown  that  potential  based  collision  avoidance  methods  arc 
the  most  control  efficient.  VV’e  then  briefly  discuss  the  dependence  of  collision 
avoidance  control  cost  on  vehicle  models.  In  particular,  for  systems  with  effi¬ 
cient  steering,  we  expect  gyroscopic  collision  avoidance  to  he  the  mast  efficient. 
We  then  show  how  collision  avoidance  can  be  used  to  randomize  surveillance  to 
give  efficient  chaotic  search  algorithms.  The  results  arc  illustrated  using  mul¬ 
tiple  underactuated  hovercraft.  Perhaps  most  interesting,  using  a  mix- norm, 
the  area  surveillance  by  multiple  hovercrafts  is  shown  to  approach  optimality 
quickly  compared  to  the  time  taken  to  survey  90%  of  the  region. 


1  Introduction 

1.1  Overview 

The  main  goal  of  this  paper  is  to  introduce  a  new  tool  for  collision  avoidance  and  to 
demonstrate  the  role  of  collision  avoidance  in  randomizing  surveillance  using  multi¬ 
ple  vehicles.  We  will  also  discuss  quantities  which  can  be  used  to  evaluate  quality  of 

*This  work  was  in  part  supported  by  DARPA  DSO  under  AFOSR  contract  FA9550-07-C-0024. 
Approved  for  public  release,  distribution  unlimited. 
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1.1  Overview  2 


surveillance  heuristics.  In  particular,  we  introduce  a  new  kinetic  shaping  based  col¬ 
lision  avoidance  controller  and  surveillance  with  arguably  chaotic  properties,  similar 
to  the  billiard  problem  [15].  Collision  avoidance  is  critical  when  managing  multi¬ 
ple  vehicles  in  an  environment  with  obstacles.  They  have  also  been  used  recently 
[6]  to  demonstrate  their  role  in  simulating  flocking  behaviour  in  addition  to  their 
intended  task  of  avoiding  collisions.  The  traditional  potential  based  methods  [18] 
and  the  recent  gyroscopic  force  based  methods  [6]  can  be  thought  of  as  shaping 
the  potential  and  the  linear  part  of  kinetic  energy  of  the  Lagrangian.  In  this  paper, 
we  complete  the  story  of  energy  shaping  based  collision  avoidance  schemes  by  de¬ 
signing  a  collision  avoidance  controller  by  shaping  the  kinetic  energy  of  the  system. 
Using  an  example  vehicle  model  and  a  scenario,  we  compare  how  the  new  technique 
fares  with  the  previous  two  methods  by  evaluating  the  the  and  norm  of  the 
control  effort.  The  former  norm  quantifies  the  fuel  consumed  and  the  latter  norm 
quantifies  the  maximum  acceleration  the  vehicle  experiences  in  avoiding  collision. 
We  also  note  that  kinetic  shaping  collision  avoidance  gives  analytic  guarantee  for  no 
collision  under  a  mild  assumption  depending  only  upon  the  energy  and  momentum 
of  the  system  as  it  enters  into  a  collision  avoidance  mode. 

Surveillance  using  multiple  vehicles  is  an  important  engineering  problem  with 
widespread  applications.  Traditionally,  this  task  has  been  categorized  into  the  static 
surveillance  problem  and  dynamic  surveillance  problems.  In  the  former,  one  needs 
to  find  the  most  optimal  placement  for  sensors  or  cameras  in  an  environment  to 
maximize  the  area  coverage  [2,  8].  In  the  latter  problem,  one  has  sensors  on  dynamic 
objects  like  vehicles  and  maybe  moving  targets  [3,  9].  The  problem  now  is  to  design 
surveillance  heuristics  to  guarantee  target  capture  and  tracking,  i.e.,  to  search  and 
secure.  The  main  research  areas  that  come  under  dynamic  surveillance  are  searcher 
coordination,  flocking/formation  control,  role  of  communication  topology  and  line  of 
sight  maintenance.  Please  see  [11]  for  a  detailed  overview  on  the  existing  literature 
on  these  topics.  In  searcher  coordination,  the  problem  is  to  make  sure  no  part  of  the 
region  of  interest  is  left  out.  The  problem  of  searching  for  missing  people  trapped  in 
mines  falls  in  this  category.  In  flocking  and  formation  control,  the  goal  is  to  make 
a  group  of  vehicles  move  in  a  group  to  achieve  a  particular  task  like  adaptive  ocean 
sampling  [7].  Whenever  one  has  multiple  vehicles,  ensuring  that  the  controller  is 
robust  to  communication  link  losses  is  of  upmost  important.  These  problems  fall 
under  the  communication  topology  category.  The  main  aim  is  to  demonstrate  that 
whatever  strategy  one  comes  up  with,  the  surveillance  is  not  compromised  because 
of  noise  or  communication  link  breakages  etc.  Finally,  when  one  has  radio  signals 
for  intervehicle  communication,  it  becomes  important  to  make  sure  line  of  sight 
is  maintained.  See  [11]  for  more  references  on  this  problem.  In  this  paper,  our 
focus  will  be  on  surveillance,  i.e.,  we  will  not  consider  communication  issues.  Our 
main  goal  is  to  make  sure  the  region  is  surveyed  and  that  too  in  a  “chaotic*’  or 
randomized  manner.  Such  randomized  searches  are  important  in  problems  where 
one  has  mobile  or  “intelligent”  targets  and  we  do  not  want  the  targets  to  predict  the 
searchers  path  for  evasion  purposes  [16].  In  our  case,  we  want  our  search  effort  to 
be  distributed  with  respect  to  some  probability  density  depending  upon  the  apriori 


50 


A.3.  COLLISION  AVOIDANCE  AND  SURVEILLANCE  WITH  AUTONOMOUS  VEHICLES 


1.2  Organization  3 


belief  for  target  distribution  and  at  the  same  time  appear  “diaotio’  enough  to  keep 
the  targets  from  predicting  the  next  location  of  the  sensors.  We  will  present  our 
surveillance  algorithm  and  demonstrate  this  property  by  comparing  the  mix-norm  of 
our  vehicle  trajectories  with  those  of  ideal  zero  inertia  vehicles  used  as  a  benchmark. 

1.2  Organization 

In  §2,  we  introduce  kinetic  shaping  collision  avoidance  controller  and  give  an  explicit 
formula  for  the  case  when  the  vehicle  is  a  double  integrator.  In  §2.2,  we  compare 
the  controller  cost  for  the  kinetic  shaping  controller  introduced  in  §2  with  the  tra¬ 
ditional  potential  based  controller  and  with  gyroscopic  forcing  based  controller  and 
interpret  the  results.  In  §3,  we  show  how  collision  avoidance  enhances  surveillance 
and  quantify  its  performance.  In  the  same  section,  we  discuss  ideal  vehicles  and  the 
mix-norm.  In  §4,  we  present  some  simulation  results  for  the  case  of  multiple  under¬ 
actuated  hovercraft  surveying  a  circular  region.  We  demonstrate  that  using  collision 
avoidance,  the  hovercraft  trajectories  approach  a  uniform  distribution  quickly  com¬ 
pared  to  the  time  taken  to  survey  90%  of  the  region.  We  conclude  this  paper  with 
a  summary  of  the  main  results  and  a  discussion  in  §o. 

2  Kinetic  Shaping  for  Collision  Avoidance 

Intervehicle  collision  avoidance  and  obstacle  avoidance  is  one  of  the  mast  impor¬ 
tant  issues  in  multivehicle  tasks.  The  vast  amount  of  literature  focussing  on  this 
particular  task  bears  testimony  to  this  claim.  Traditional  methods  based  on  po¬ 
tential  design  can  be  traced  to  the  original  works  of  Rimon  and  Koditschek  [18]. 
The  basic  idea  is  to  assign  repelling  potentials  to  obstacles  and  other  vehicles  and 
shape  the  potential  energy  of  the  moving  vehicle  accordingly.  To  get  rid  of  some 
of  the  drawbacks  of  potential  based  collision  avoidance,  viz,  global  knowledge  of 
obstacles  etc,  the  authors  in  [6]  design  gyroscopic  forcing  based  collision  avoidance 
schemes.  One  of  the  main  advantages  of  this  new  approach  is  that  it  tends  to  avoid 
gridlocking.  Moreover,  it  conserves  the  total  energy  of  the  system  as  the  steering 
forces  act  perpendicular  to  the  instantaneous  velocity.  This  in  turn  implies  that  we 
can  use  the  readymade  energy  function  of  the  system  as  a  Lyapunov  function  for 
stability  analysis  purposes.  Moreover,  the  gyroscopic  forces  do  not  interfere  with 
the  global  potential  function  designed  for  a  particular  control  task.  The  gyroscopic 
force  based  collision  avoidance  is  also  completely  decentralized.  Each  vehicle  has  a 
sensing  radius  and  goes  into  a  collision  avoidance  mode  only  when  it  senses  other 
vehicles  within  its  sensing  radius. 

Gyroscopic  based  collision  avoidance  discussed  in  [6]  can  be  thought  of  shaping 
the  Lagrangian  by  introducing  a  term  linear  in  velocity.  For  example,  if  {qi)  = 
q  are  generalized  coordinates,  then  we  can  introduce  gyroscopic  forces  into  the 
system  by  adding  the  term  G  =  to  the  Lagrangian*.  The  gyroscopic 

forcing  term  responsible  for  collision  avoidance  acts  in  a  direction  “perpendicular” 

intrinsically,  wc  arc  adding  the  action  of  the  one  form  A  =  A,dq'  on  the  tangent  bundle  to  the 
Lagrangian. 
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to  the  instantaneous  velocity.  What  this  means  is  that  the  Euler- Lagrange  term 
corresponding  to  G  adds  the  term  ~  to  the  right  hand  side  of  the 

Euler-Lagrange  equation  for  the  original  Lagrangian  in  the  direction.  It  can  be 
verified  that  the  dot  product  of  this  term  with  q  is  zero,  i.e.,  the  gyroscopic  forcing 
term  is  “perpendicular*’  to  the  instantaneous  velocity.  Note  also  that  if  the  one 
form  Aidq^  is  closed,  the  gyroscopic  forcing  term  is  zero.  Potential  based  collision 
avoidance  discussed  in  [14]  also  shapes  the  potential  energy  part  of  the  Lagrangian. 
But  now,  the  resulting  collision  avoidance  forcing  term  can  be  made  to  act  either 
along  the  center  lines  of  the  vehicles  as  in  [18],  or  in  a  direction  perpendicular 
to  the  centerline  or  a  combination  of  these  two  directions.  In  the  same  paper,  it 
was  observed  in  simulation  examples  that  when  the  forcing  term  acts  in  a  direction 
perpendicular  to  the  centerline,  it  takes  a  much  longer  time  for  the  vehicles  to  get 
out  of  collision  avoidance  mode.  After  studying  the  above  two  examples,  one  could 
ask  if  its  possible  to  derive  collision  avoidance  schemes  by  shaping  the  kinetic  energy 
part  of  the  Lagrangian.  We  provide  an  answer  in  positive  in  this  section. 

The  model  we  will  be  using  in  this  section  is  going  to  be 

X  =  Ur -f  Ur -f  u/)  (2.1) 

where  ur  is  the  controller  which  takes  the  vehicles  to  a  particular  target  location 
or  achieve  a  particular  task,  uc  is  the  collision  avoidance  controller  and  up  is  the 
dissipation  controller  linear  in  velocity.  Each  vehicle  has  a  sensing  radius  and  a 
collision  radius  rc,  i.e.,  a  vehicle  senses  another  vehicles  which  is  within  a  distance 
Vs  from  it.  If  the  distance  between  two  vehicles  is  less  than  Tc,  its  considered  a 
collision.  So  our  goal  is  to  derive  a  collision  avoidance  scheme  which  guarantees  that 
the  distance  between  two  vehicles  is  always  greater  than  2rc.  The  collision  avoidance 
controller  uc  is  nonzero  only  when  the  vehicle  senses  an  obstacle  or  another  vehicle 
inside  its  sensing  radius.  When  the  vehicle  is  in  its  collision  avoidance  mode,  ur 
and  up  are  both  set  to  zero. 


Figure  2.1:  Vehicle  j  within  the  radius  of  sensing  r.,  of  vehicle  i. 
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Consider  two  vehicles  in  R”  w'hose  controlled  equations  of  motion  are 

X,  =  uri  +  uci  +  UDi  (2.2) 

X2  =  Ur2  +  UC2  +  UD2  (2.3) 

The  Lagrangian  for  each  of  these  vehicles  are  given  by  their  respective  kinetic  en¬ 
ergies  Li  =  ^Xi^Xi.  As  stated  before,  when  vehicles  i  senses  another  vehicle  within 
its  tolerance  radius,  it  goes  into  a  collision  avoidance  mode  where 

UTi  =  0;  uoi  =  0  (2.4) 

In  this  collision  avoidance  mode,  we  would  now  like  to  choose  the  collision  avoid¬ 
ance  controller  uci  such  that  the  resulting  closed  loop  system  consisting  of  the  two 
vehicles  is  also  a  Lagrangian  system  with  a  Lagrangian  which  looks  like  a  kinetic 
energy  term.  Consider  the  Lagrangian  L  given  by 

^l|xi -X2f(x?  +  x^)  (2.5) 

This  Lagrangian  is  regular  as  long  as  xi  ^x^.  The  energy  E  corresponding  to  this 
Lagrangian  is  the  Lagrangian  itself  and  the  momentum  corresponding  to  translation 
symmetry  is  p  =  ||xi  —  X2p(xi  +X2).  We  will  show  that  for  two  vehicles  following 
the  Euler-Lagrange  equations  for  this  Lagrangian,  the  vehicles  do  not  collide  as  long 
as  their  initial  velocities  are  not  equal  and  opposite  to  each  other.  More  precisely, 
we  have  the  following  result. 

Theorem  Consider  two  vehicles  with  configuration  variables  given  by  xi,X2  and 
following  the  equations  of  motion  corresponding  to  the  Lagrangian  given  by  (2.5). 
If  at  i  =  0,  we  have  xi(0)  ^  X2(0)  and  xi(0)  +  X2(0)  ^  0,  then  the  two  vehicles 
never  collide  at  any  instant  of  time,  i.e.,  xi{t)  ^  X2{t)  for  all  t  >  0.  Moreover,  if 
Eo  >  0  and  Po  ^  0  are  the  nonzero  energy  and  momentum  at  i  =  0  respectively, 

then  II  xi  -  X2II  >  for  all  time. 

PROOF  For  the  Lagrangian  given  by  (2.5),  the  momentum  p  corresponding  to 
translation  symmetry  and  the  energy  E  are  conserved  quantities.  Assume  that 
11^1  “  ^2||  >  0*  Then  we  have 

Eo  =  ^||xi  -  X2f  [||xi  +X2f  +  ||xi  -X2f] 

>  jl|xi  -X2f  ||X1  +X2||^ 

^  I  ||x! -°X2|P  (  Assuming  ||xi -X2II  >0)  (2.6) 

Therefore,  we  get  that  if  ||xi  —  X2II  >  0,  then  ||xi  —  X2II  >  >  0.  Thus,  using 

continuity  and  the  fact  that  xi(0)  /  X2(0),  wt  get  that  the  minimum  distance  of 
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approach  for  the  vehicles  is  bounded  below  by  a  quantity  depending  only  on  the 
values  of  energy  and  momentum  at  i  =  0.  ■ 

If  we  now  use  the  equations  of  motion  given  by  the  Euler  Lagrange  equation 
corresponding  to  the  Lagrangian  (2.5),  we  have  designed  a  collision  avoidance  con¬ 
troller  which  guarantees  no  collision  as  long  as  \/W  >  Tc.  Here,  subscript  s 
denotes  the  instance  when  the  vehicles  are  within  each  others  sensing  radius  and 
have  sensed  each  other.  At  this  instant,  the  only  controller  acting  on  the  vehicles 
are  the  individual  collision  avoidance  contoller  derived  from  the  Lagrangin  (2.5). 
The  final  controller  for  the  individual  vehicle  turns  out  to  be 


uci  = 

UC2  = 


(xi  ~X2)^  (“2(Xl  -  X2)^(Xi  -  X2)Xl  +  (xj  +  x^)(xi  -  X2)) 
(xi  ^X2)^  (“2(xi  -  X2)^(Xi  -  X2)X2  +  (x?  +  X^)(X2  -  Xj)) 


(2.7) 


Figure  (2.2)  illustrates  trajectories  in  the  (x\,X2)  plane  for  two  vehicles  in  R 
with  coordinates  x\^X2.  The  initial  position  is  (0.5,  0)  and  various  trajectories 
correspond  to  various  initial  velocities  starting  from  this  particular  initial  position. 
As  shown,  the  trajectories  never  “collide*’  with  the  line  x\  =  X2  unless  its  initial 
momentum  is  zero  in  which  case  they  collide  in  finite  time.  We  have  the  following 
result. 


Figure  2.2:  Trajoctorics  in  (xi,X2)  plane  for  two  vehicles  in  R  with  positions  given  by  Xi,X2 
and  corresponding  to  the  Lagrangian  given  by  (2.5).  The  initial  position  is  (0.5, 0)  and  the  initial 
veloeities  are  shown  in  the  figure. 


Result  Consider  two  vehicles  in  R  with  positions  given  by  X\yX2  and  following 
the  equations  of  motion  given  by  the  Lagrangian  (2.5).  If  i:i(0)  +  X2(0)  =  0  and 
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xi(0)  ^  0:2(0),  then 

(2.8) 

where  z\{t)  =  xi(^)  —  X2{t)  and  2i(0)  ^  0  by  ai>sumption.  Therefore,  we  have 
collision  in  finite  time. 

PROOF  Proof  follows  by  writing  down  the  equations  of  motion  for  the  Lagrangian 
(2.5)  in  R  using  the  variables 


Z\  =  Xi  —  X2 

Z2  =  X'i  +  X2  (2.9) 


and  verifying  (2.8).  ■ 

We  also  have  the  following  time  to  minimum  approach  result. 

Result  Consider  two  vehicles  in  R  with  positions  given  by  xi,X2  and  following 
the  equations  of  motion  given  by  the  Lagrangian  (2.5),  If  £;  =  2L(0)  >  0  and 
p  =  (a:i(0)  —  X2(0))^  (^i(O)  +  2^2(0))  ^  0,  the  the  vehicles  approach  their  minimum 
distance  at  time  ?  where 


C2 


{2\/a(z2(0)  —  k)  +  sinh(2v/a(22(0)  —  k))) 


Cl  =  -2\/az2(0)  (1  +  cosh(2v/a(22(0)  —  k)) 

where  Z\yZ2  are  as  defined  above  in  (2.9)  and  k  =  —  «2:2(0)- 

PROOF  We  have, 

L  =  7^i(ii+i|) 


P  = 


2 1 22 


(2.10) 

(2.11) 


If  £  >  0  and  p  ^  0,  we  have  ^  — T.  This  gives,  =  -^  cosh(v/a(22  — 

k)).  Therefore,  ii  =  sinh(v/a(2:2  “^"))22  and  22  =  ’-2v/a  tanh(\/Q(22  —  ^))i2'  There¬ 
fore, 

(2.12) 


where 


Z2(t)  =  k-h  — =Root(sinh(2)  -h  2  —  4(^ci  -h  C2)  =  0) 
2y  a 


C2  =  -  (2y/a(z2(0)  —  k)  -h  sinh(2v/a(22(0)  —  k))) 

Cl  =  ^2v/a22(0)  (1  -h  cosh(2\/a(22(0)  —k)) 


(2.13) 

(2.14) 


Now,  2i  is  minimum  when  22  =  A:  at  ^  =  Tmin-  This  implies, 


= 
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2.1  Comparison  with  potential  and  gyroscopic  forcing  based  con¬ 
trollers 

In  this  section,  we  will  compare  the  performance  of  kinetic  shaping  collision  avoid¬ 
ance  controller  with  the  traditional  potential  shaping  based  and  the  recent  gyro¬ 
scopic  shaping  based  controller.  Figure  2.3  compares  qualitatively  the  difference  in 
trajectories  for  the  three  different  collision  avoidance  schemes.  In  this  figure,  we 
have  two  double  integrator  vehicles  in  R^,  initially  located  at  (30,  0)  and  (—30,  0). 
Starting  with  zero  initial  velocities,  their  task  is  to  swap  their  positions  avoiding 
collisions.  The  controllers  will  be  stated  precisely  in  (2.17).  The  plot  illustrates 
the  trajectories  as  the  vehicles  swap  their  positions.  We  see  that  qualitatively,  the 
trajectories  for  collision  avoidance  based  on  gjToscopic  forcing  and  kinetic  shaping 
are  comparable.  Whereas  in  the  traditional  potential  forcing  based  controller,  we 
see  the  typical  spring-mass  like  bouncing  back  phenomena.  We  also  see  that  gyro¬ 
scopic  based  collision  avoidance  starts  steering  the  vehicles  away  from  each  other 
earlier  on  and  performs  a  less  aggressive  maneuver  compared  to  kinetic  shaping 
based  controller. 


Figure  2.3:  Two  vehicles  swapping  their  positions.  The  three  trajectories  correspond  to  three 
different  collision  avoidance  schemes.  As  can  be  seen,  the  trajectories  for  kinetic  shaping  and 
gyroscopic  forcing  based  collision  avoidance  arc  qualitatively  comparable. 


Collision  avoidance  when  n  >  2  When  we  have  more  than  two  vehicles,  there 
are  at  least  two  strategies  to  handle  this  situation  by  essentially  reducing  it  to  a  two 
vehicle  problem.  In  the  first  case,  each  vehicle  detects  all  the  other  vehicles  within 
its  sensor  radius  and  treats  the  average  state  of  its  neighbours  as  another  vehicle, 
thus  reducing  the  multiple  neighbour  problem  to  a  two  body  problem.  This  strategy 
is  not  new  and  seems  to  work  in  simulations.  See  [6]  for  example  and  references 
therein.  The  second  alternative  is  for  each  vehicle  to  avoid  collision  with  its  nearest 
neighbour  amongst.  We  will  use  the  latter  in  all  the  simulations  in  this  paper.  It 
also  appears  that  kinetic  shaping  based  controller  can  be  extended  to  n  >  2  case  by 
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considering  a  Lagrangian  of  the  form 

^  =  l  +  =<?))  (2-15) 

^  l<i<j<n 

without  resorting  to  any  of  the  heuristics  discussed  above.  We  will  not  be  considering 
this  scenario  in  this  paper  and  rather  focus  on  avoiding  collision  with  the  nearest 
neighbor  and  compare  the  performances  between  kinetic,  potential  and  gyroscopic 
controllers. 


2.2  Cost  comparison  with  potential  and  gyroscopic  forcing  based 
controllers 


Figure  2.4:  TYajectorics  of  20  vehicles,  each  starting  on  the  circumference  and  reaching  its  assigned 
target  location  with  collision  avoidance  based  on  kinetic  shaping  controller. 


We  now  use  a  specific  task  to  compare  how  our  kinetic  shaping  based  colllision 
avoidance  compares  with  the  potential  and  gyroscopic  based  controller.  We  choose 
the  following  scenario  and  use  two  norms  to  do  the  comparison.  Consider  n  vehicles, 
each  with  a  sensor  radius  of  2m,  initial  distibuted  at  an  equal  spacing  on  the  circum¬ 
ference  of  a  circle  with  radius  7m.  We  vary  n  from  10  to  20.  At  i  =  0,  these  vehicles 
are  assigned  a  random  target  position,  again  lying  on  the  circumference.  Figure  2.4 
shows  a  sample  trajectory.  The  plot  on  the  left  shows  the  initial  configuration  and 
the  plot  on  the  right  shows  the  projection  of  the  trajectories  in  time  onto  the  plane 
after  the  vehicles  have  reached  their  respective  target  locations. 

We  choose  the  three  different  kinds  of  collision  avoidance  controller  as  discussed 
above  to  evaluate  the  performances  of  each.  Specifically,  we  compare  the  cost  func¬ 
tions  given  by  and  ||u||oo  and  norm  of  u)  over  20  runs  for  each  value 

of  n  and  each  collision  avoidance  scheme.  Here,  u  is  the  sum  ut  H-  uc  +  ud.  The 
former  norm  quantifies  the  amount  of  fuel  consumed  and  the  latter  norm  quantifies 
the  maximum  acceleration  experienced  by  the  vehicles.  We  illustrate  the  results  in 
Figures  2.5,  2.6  and  2.7. 

For  the  cost  comparison,  we  choose  the  following  controllers. 


UTi  =  -krixi  -  XTi) 
UDi  =  -koXi 


(2.16) 
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Here,  xxi  is  the  target  point  sussigned  to  the  vehicle,  kr  is  the  potential  gain 
which  drives  the  vehicle  to  its  target  and  ko  is  the  dissipation  gain.  The  only 
controller  that  varies  is  the  collision  avoidance  controller  uct*.  Let  us  denote  the 
state  of  the  nearest  neighbour  of  the  vehicle  by  x/vi,x/vi-  Then,  we  have  the 
following  expression  for  the  collision  avoidance  controller  for  the  vehicle. 


_  (-2(xi  -  XNif(x.i  -  x^i)Xi  +  (xf  +  x^,-)(xi  -  XNi)) 

(x,-  -  Xwf)2 
OR 

=  fcpc exp(ii- — 5- — ji)(x,  -  XNi) 

OR 

=  fcgcexp(  ^  ■  )A^x, 

||Xi  -  XNill 


(2.17a) 


(2.17b) 


(2.17c) 


Here,  (2.17a),(2.17b)  and  (2.17c)  correspond  to  kinetic  shaping,  potential  shaping 
and  gyroscopic  shaping  based  collision  avoidance  controllers  respectively,  kpc^kgc 
are  controller  gains  and  K  is  a  constant  skew  symmetric  matrix.  Please  see  [14]  for 
more  details  on  the  potential  and  gyroscropic  controllers  in  (2.17).  The  parameters 
kryko  in  (2.16)  are  fixed  and  kpc^kgc  in  (2.17)  are  chosen  so  as  to  keep  the  average 
minimum  distances  over  20  simulations  to  be  around  2%  of  the  sensor  radius.  This 
is  illustrated  in  Figure  2.5.  In  our  case  of  planar  vehicles,  the  sensor  radius  is  2m 
and  K  is  chosen  to  be 


K  = 


0  1 

-1  0 


(2.18) 


As  can  be  seen  in  Figure  2.6,  for  the  gains  corresponding  to  the  minimum  dis¬ 
tances  in  Figure  2.5,  the  collision  avoidance  based  on  potentials  seems  to  have  the 
least  average  control  effort.  Both,  the  kinetic  based  and  gyroscopic  based  collision 
avoidance  are  comparable  to  each  other  and  have  a  similar  pattern.  Even  when  one 
looks  at  the  maximum  acceleration  norm  ||u||oo,  Figure  2.7  illustrates,  collision 
avoidance  based  on  potentials  has  the  least  maximum  acceleration.  From  the  above 
discussion,  it  appears  that  atleast  for  the  particular  task  we  have  considered  and  the 
particular  double  integrator  model  for  the  vehicles,  potential  based  collision  avoid¬ 
ance  seems  to  be  the  least  expensive.  Ofcourse,  for  different  vehicle  models,  one 
needs  to  do  a  similar  study  and  make  conclusions.  In  our  double  integrator  vehicle 
model,  thrusting  and  steering  were  both  equally  expensive.  For  vehicles  which  have 
a  cheaper  way  of  gyroscopic  steering  as  compared  to  ours,  we  expect  gyroscopic 
based  collision  avoidance  to  perform  better.  One  also  needs  to  be  a  bit  more  careful 
when  making  these  comparison.  For  example,  it  is  known  that  using  potential  based 
techniques,  it  is  observed  that  vehicles  do  get  stuck  in  loc^l  extremums  [6]. 
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Figure  2.5:  The  parameters  in  (2.IG)  and  (2.17)  are  chosen  sueh  that  the  average  minimum 
distance  over  20  runs  which  any  two  vehicles  approach  are  comparable  for  all  the  three  strategics 
based  on  kinetic,  potential  and  gyroscopic  collision  avoidance.  The  sensor  radius  for  the  vehicles  in 
the  numerical  simulation  arc  all  2m. 
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3  Randomized  Area  Surveillance  Using  Collision  Avoid¬ 
ance 

In  this  section,  we  will  demonstrate  how  collision  avoidance  can  be  used  to  enhance 
area  surveillance  by  “randomizing’  them.  VV"e  also  make  sure  this  is  done  in  a 
decentralized  and  scalable  manner  and  is  independent  of  the  search  domain  and 
its  topology.  The  main  motivation  for  such  search  strategies  is  to  make  sure  that 
an  adversary  target  is  not  able  to  predict  the  state  of  search  vehicles  in  order  to 
dodge  them.  This  is  in  stark  contrast  to  the  conventional  “lawn mover”  techniques 
in  which  case  a  target  r.an  avoid  detection  by  just  following  the  lawnmovers.  Some 
typical  applications  of  randomized  search  strategies  or  policies  are  for  example  police 
patrolling  and  airport  security  systems  [Hi]. 

The  mix-norm  VVe  now  make  precise  what  we  mean  by  “randomizing”  searches. 
Our  main  interest  is  two  fold.  One  is  to  make  sure  the  whole  region  is  swept  without 
any  missed  spots.  Second,  we  need  to  do  this  such  that  the  vehicle  trajectories 
appear  random  to  the  targets.  For  the  first  case,  the  problem  is  more  interesting  in 
the  case  when  the  ratio  of  sensing  radius  to  the  are^  of  the  region  is  arbitrarily  small. 
In  this  case,  in  order  to  quantify  randomization,  we  use  the  mix-norm  introduced 
in  [13]  in  the  context  of  fluid  mixing.  In  the  same  paper,  a  scalar  field  is  said  to 
be  well-mixed  if  its  averages  over  arbitrary  open  sets  are  uniform.  For  the  vehicle 
trajectory  case,  the  basic  idea  is  the  following.  We  say  that  the  trajectory  of  a 
vehicle  is  uniformly  distributed  in  a  region  if  the  time  average  of  a  function 
over  the  trajectory  asymptotically  approaches  its  space  average  with  respect  to  the 
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Figure  2.6:  Plot  comparing  the  total  control  cost  to  achieve  a  task  using  three  kinds  of  collision 
avoidance,  kinetic,  potential  and  gyroscopic  based.  As  can  be  seen,  collision  avoidance  based  on 
potentials  seems  to  be  having  the  least  control  effort. 

uniform  distribution  ii.  It  is  known  that  this  is  a  necessary  and  sufficient  condition 
for  ergodicity  [17].  If  \Vt  is  the  distribution  whicli  takes  a  function  and  evaluates 
its  mean  over  the  trajectory  upto  time  then  we  demonstrate  that  once  collision 
avoidance  is  introduced  into  the  system,  the  distribution  Wt  ap proaces  /z  when  t  for 
t  much  smaller  than  the  time  taken  for  the  vehicles  to  survey  90%  of  the  region. 
This  is  the  sense  in  which  our  search  is  optimally  randomized.  Once  the  trajectories 
start  approximating  a  uniform  distribution,  we  expect  the  targets  not  being  able  to 
predict  the  future  behavior  of  the  vehicles. 

As  an  illustration,  consider  the  plots  in  Figure  3.1  and  Figure  3.2.  The  first 
figure  depicts  Hilbert  curves  for  indices  n  from  1  to  4  which  are  space  filling  curves 
in  the  limit  n  — oo.  And  Figure  (3.2)  is  a  plot  of  the  decay  of  mix-norm  over  these 
curves.  We  see  that  as  the  index  increases,  i.e.,  as  the  Hilbert  curves  gets  closer  to 
a  space  filling  curve,  the  mix-norm  decays  at  a  faster  rate. 

Benchmark  vehicles  Another  way  to  see  randomness  is  the  following.  The  best 
surveillance  one  can  achieve  is  using  ideal  vehicles  with  zero  inertia  hoping  around 
instantaneously  in  the  region  such  that  their  discrete  trajectory  points  come  from 
a  uniform  distribution.  We  are  essentially  comparing  our  vehicles  with  these  ideal 
ones.  When  we  say  ITj  approximates  a  uniform  distribution,  we  mean  that  mean  of 
samples  of  a  function  along  the  trajectory  approaches  the  mean  of  samples  of  the 
same  function  along  the  same  number  of  points  chosen  from  a  uniform  distribution. 

Note  that  if  we  have  apriori  knowledge  of  target  locations  represented  by  a 
distribution,  we  can  exploit  this  information  further  by  combining  collision  avoidance 
with  random  way  point  assignment  which  are  chosen  from  the  apriori.  In  this 
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Figure  2.7:  Plot  of  the  maximum  acceleration  a  vehicle  experiences  versus  the  total  number  of 
vehicles  in  our  task.  Agmn,  one  can  see  that  collision  avoidance  based  on  potentials  has  the  least 
maximum  acceleration. 

setting,  we  have  a  decentralized  collision  avoidance  and  a  centralized  “manager"’, 
which  keeps  updating  the  waypoints  for  inidividual  vehicles.  The  vehicles  are  free 
to  decide  how  they  approach  the  waypoints  and  what  kind  of  collision  avoidance 
they  want  to  adopt.  This  is  rcmiiisccnt  of  the  inner  and  outer  loop  philosophy  in 
[10]  for  example.  We  briefly  discuss  this  setting  below. 

Problem  Setting  With  Apriori  Knowledge  We  have  a  region  A,  with  maybe 
obstacles,  which  needs  to  be  surveyed.  The  region  A  also  has  a  probability  distri¬ 
bution  for  events,  i.e.,  there  could  be  areas  within  A  where  events  are  more  likely 
to  happen.  We  also  have  a  finite  number  of  vehicles  with  a  finite  sensor  radius. 
Consider  a  lattice  L  with  I  points  which  reflects  the  probability  distribution  for 
the  events.  See  the  figure  below  which  illustrates  a  lattice  for  uniform  and  Gaussian 
probability  distributions  respectively  for  the  case  when  the  region  A  is  circular.  The 
lattice  spacing  is  a  function  of  the  individual  sensor  radius. 

Consider  Algorithm  1  below.  The  vehicles,  labelled  1  to  n  are  initially  in  some 
arbitrary  locations.  We  assume  that  each  vehicle  has  a  set  point  controller  which 
asymptotically  takes  it  to  a  particular  target  location. 


Algorithm  1  Surveillance  Algorithm  A 
1:  Assign  n  random  points  from  L  as  target  positions  to  the  vehicles 
2;  Once  the  n  vehicles  reach  their  individual  point,  select  new  points  from  L  and 
assign  them  as  target  point  to  the  vehicles 
3;  Repeat  this  procedure  until  L  is  exhausted 


Throughout  the  steps  above,  we  incorporate  collision  avoidance  into  the  setting, 
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Figure  3.1'.  Illustration  of  Hilbert  curves  for  indexes  running  from  1  to  4 

i.e.,  if  in  the  process  of  reaching  their  respective  target  point  two  or  more  vehicles 
come  close  to  each  other,  they  go  into  a  collision  avoidance  mode.  This  strategy 
has  some  advantages  over  the  traditional  lawnmover  strategies.  The  manager  only 
assigns  target  positions  to  the  individual  vehicles  as  opposed  to  complete  individ¬ 
ual  trajectory.  Thus,  we  have  a  target  assignment  problem  instead  of  a  trajectory 
generation  problem  which  works  for  a  much  broader  class  of  regions.  This  is  be¬ 
cause  generating  lawnmover  trajectories  for  arbitrary  regions  is  nontrivial  [1].  Our 
method  also  has  advantages  when  considering  underactuated  vehicles,  for  which  we 
only  need  to  design  a  set  point  controller  as  opposed  to  making  them  follow  a  partic¬ 
ular  trajectory.  The  individual  vehicles  are  free  to  choose  how  to  reach  their  target 
positions  providing  the  freedom  to  chose  optimal  controllers  for  example. 

It  can  be  numerically  verified  that  with  this  strategy  the  trajectories  start  ap¬ 
proximating  a  uniform  distribution  quickly  compared  to  the  time  taken  to  survey 
90%  of  the  region.  Note  that  in  this  case,  it  is  not  very  clear  where  this  random¬ 
ization  is  coming  from.  We  have  a  combination  of  random  waypoint  assignments 
as  well  as  cx^llision  avoidance.  To  resolve  this  and  to  make  precise  the  role  of  colli¬ 
sion  avoidance,  we  instead  consider  the  following  scenario  where  there  is  no  apriori 
knowledge. 

Problem  Setting  Without  Apriori  Knowledge  We  have  a  region  A,  with 
maybe  obstacles,  which  needs  to  be  surveyed  and  consider  two  cases.  In  the  first 
case,  one  starts  with  random  positions  for  the  n  vehicles  and  assigns  them  random 
initial  velocities.  This  essentially  determines  the  future  of  all  the  vehicles  as  the 
survey  the  area  and  we  compute  the  mix  norm  of  the  trajectories.  In  the  second 
case,  one  incorporates  collision  avoidance  into  the  first  case  and  again  evaluates 
the  mix  norm  of  the  trajectories.  It  turns  out  that  once  collision  avoidance  is 
incorporated,  the  mix  norm  decays  at  a  much  faster  rate  thus  validating  our  claim 
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Figure  3.2:  Plot  of  decay  of  mix-norm  over  the  hilbert  curves  for  the  indices  in  Figure  3.1. 


Figure  3.3:  Illustration  of  a  region  A  with  a  grid  reflecting  uniform  and  Gaussian  probability 
distribution  for  target  locations. 


that  collision  avoidance  enhances  the  surveillance  by  randomizing  it. 

Cumulative  Area  Coverage  Computation  To  calculate  the  area  covered,  we 
distribute  points  from  a  uniform  distribution  in  A  and  label  them  red.  When  a 
vehicle  moves  over  it,  we  turn  it  green.  Then  the  cumulative  area  coverage  (CAC) 
is  the  ratio  of  green  points  to  the  total  number  of  points.  This  way,  we  can  take 
care  of  a  broad  class  of  surveillance  regions  and  sensor  sensing  profiles.  For  example, 
calculating  the  exact  area  occupied  by  a  finite  number  of  circles  with  varying  radii 
involves  a  nontrivial  implementation  of  the  inclusion  exclusion  principle. 

4  Simulation 

We  now  apply  our  surveillance  strategy  using  vehicles  which  are  simplified  models 
for  a  hovercraft.  The  concrete  situation  is  the  following.  Consider  a  circular  region 
A  with  radius  50  m  and  8  underactuated  identical  hovercraft  as  illustrated  in  Figure 
•1.1  with  the  following  Lagrangian. 

L  =  ^(mx^ -h  my^ -h  (4.1) 
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Tlie  control  force  fi  acts  on  the  center  of  mass  and  the  control  torque  /2  acts  on  the 
body  at  a  distance  r  from  the  center  of  mass  as  shown  in  Figure  1.1,  The  equations 
of  motion  can  be  derived  to  be 

/  mx  \  /  cos(0)/i  -  sin(0)/2  \ 

I  'ny  I  =  (  sin(6()/i  +  cos(6»)/2  j  .  (4.2) 

In  Appendix  A,  we  show  how  to  choose  /i,/2  to  design  set  point  controller,  dissi¬ 
pative  controller  and  gyroscopic  force  based  collision  avoidance  controller  for  this 
particular  hovercraft  system.  Using  these  controllers,  we  implement  our  surveillance 
strategy  with  collision  avoidance. 


Figure  4.2:  Plot  illustrating  surveillance  enhancement  with  collision  avoidance. 

As  shown  in  Figures  1.1  and  1.3,  for  our  algorithm,  the  vehicles  take  about  30 
seconds  to  survey  90%  of  the  circular  region  and  it  takes  only  about  2  seconds  for 
the  trajectories  to  approximate  a  uniform  distribution.  Please  see  Appendix  B  for 
details  on  how  we  compute  the  mix-norm  for  the  hovercraft  trajectories. 
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Figure  4.3:  Plot  of  minx- norm  of  the  vehicle  trajectories  and  those  of  a  uniform  distribution. 
As  can  be  seen,  the  two  norms  match  pretty  well  after  about  1  see  even  though  the  time  taken  to 
survey  90%  of  the  area  is  around  30  sec. 


5  Conclusions 

We  now  conclude  this  paper  with  a  summary  of  the  main  results.  We  introduced 
a  kinetic  shaping  based  collision  avoidance  and  compared  its  performance  in  the 

and  norms  with  that  of  potential  and  gyroscopic  based  collision  avoidance 
schemes.  For  the  particular  vehicle  model  we  considered,  it  turns  out  that  potential 
based  collision  avoidance  were  the  most  efficient.  For  other  vehicles  with  more  com¬ 
plicated  dynamics,  it  cx)uld  very  well  be  that  potential  based  collision  avoidance  is 
not  the  most  efficient.  For  examples,  vehicles  in  which  thrusters  are  more  expensive 
compared  to  steering,  it  is  expected  that  gyroscopic  forcing  based  collision  avoidance 
will  turn  out  to  be  cheaper. 

We  also  demonstrated  the  role  of  collision  avoidance  in  efficiently  randomizing 
surveillance  similar  to  the  billiard  problem.  The  randomization  was  quantified  using 
the  mix-norm  from  fluid  mixing  literature.  In  our  simulation,  we  showed  that  for 
our  case  of  underactuated  hovercraft,  the  trajectories  approximate  a  uniform  dis¬ 
tribution  much  earlier  than  the  time  taken  to  survey  90%  of  the  region.  We  hope 
to  analytically  prove  the  decay  of  mix-norm  for  the  hovercraft  system  in  a  future 
publication.  We  also  demonstrated  randomization  in  essentially  decentralized  man¬ 
ner.  This  is  in  contrast  with  ?  ]  where  they  achieve  uniformization  by  optimizing 
a  global  cost  function  in  a  centralized  manner.  Merging  the  work  in  ?  ]  with  ours 
will  be  an  interesting  future  direction. 
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APPENDIX  A 

6  Controllers  for  the  underactuated  hovercraft 

In  this  section,  we  will  design  potential,  dissipative  and  gyroscopic  force  based  con¬ 
troller  for  the  hovercraft  system. 
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Design  of  gyroscopic  controller  Let  5  be  a  skew  symmetric  matrix  given  by 

(0  Si  32  \ 

-Si  0  S3  j  .  (6.1) 

-S2  -S3  0  / 

For  the  right  hand  side  of  (4.2)  to  be  a  gyroscopic  force,  the  following  equation  must 
hold  true. 

/  cos(^?)/i  *-  sin(^?)/2  \  /  ^  \ 

I  sin(0)/i  -h  cos(0)/2  j  =  5  I  ^  j  .  (6.2) 

Substituting  for  S  from  (b.l)  in  (6.2)  and  comparing  the  coefficients  of  we 

get  the  following  conditions  on  entries  of  the  matrix  S. 

32  =  ‘-Sircos(^) 

S3  =  -5irsin(61). 

Therefor,  the  only  parameter  we  have  for  tuning  the  gyroscopic  force  is  S}.  We  now 
solve  for  /i,/2  in  terms  of  s\  to  get  the  final  equations  of  motion  with  gyroscopic 
forcing  as 

(mx  \  /  y  —  rcoiy{0)9  \ 

'nfiy  )  =  (  — a:  — rsin(^)^  j  .  (6.3) 

JO  )  \  r  cos(^)x  +  r  sin(0)^  / 

One  can  readily  ciieck  that  the  dot  product  of  the  right  hand  side  of  above  equation 
with  velocity  vector  is  indeed  zero.  Equation  (6.3)  represents  the  most  general 
gyroscopic  forcing  for  the  hovercraft.  For  our  simulations  in  §4,  we  choose  the 
parameter  5i  to  be  a  constant. 

Design  of  potential  controller  We  design  potential  based  setpoint  controller 
for  the  hovercraft.  Because  of  underactuation,  we  cannot  make  the  hovercraft  flow 
along  an  arbitrary  potential  field.  Instead,  we  derive  conditions  on  the  possible 
potentials  which  one  can  use  for  a  hovercraft.  In  (4.2),  assume  the  right  hand  side 
is  given  by  a  poteltial  V{x^y^0)^  i.e.. 


/  cos(0)/i  -  sin(6»)/2 
sin(<?)/i  +  cos(0)/2 

V  -r!2 


\ 

/ 


We  can  derive  the  following  condition  on  V. 


(W 

00 


^rMO)-^-rccs(0)  —  . 


(6.4) 


(6.5) 
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Using  method  of  characteristics,  we  can  show  that  the  most  general  solution  for  the 
potential  V  in  (G.5)  can  be  found  out  to  be 

V  =  V(x  —  rcos(0),2/  —  rsin(6^))  =  ^(21,^2)*  (6.6) 

Using  V  as  in  (6.6),  we  can  design  controller  which  will  asymptotically  take  the 
hovercraft  to  the  desired  set  (21,2:2)  =  (^idi^2d)'  T'his  in  particular  implies  that  we 
can  only  guarantee  that  the  hovercraft  reach  a  neighbourhood  of  a  desired  target 
point  (Xd,  yd)  in  the  plane  without  control  over  its  final  orientation  given  by  6. 

Design  of  dissipation  controller  In  this  case,  we  want  to  choose  /i,/2  such 
that 

/  cxxs(0)/i-sin(^)/2  \  f  x\ 

I  sin(^)/i  4-  cos(^)/2  j  =  -A:  (  2/  j  .  (6.7) 

One  can  easily  check  that  for  the  following  choices  for  /i,  /2,  (6.7)  is  satisfied. 

/i  =  ~k(coii{6)x -\-iim{6)y)  (6.8) 

h  =  -0  (6.9) 

r 

APPENDIX  B 

Computing  norm  Let  f{t)  €  R^,  t  6  [0,7"]  be  the  trajectory  of  the  system 
up  to  time  7".  VVe  need  to  compute  how  “close*’  this  trajectory  is  to  a  uniform  distri¬ 
bution.  To  compute  this,  we  take  N  samples  with  N  sufficiently  large,  uniformly  in 
time  given  by  f{ti)  where  i  6  {0,  •  •  •  ,  A  —  1}  and  ti  =  7^^ 7".  Denote  this  collection 
hy  Ss  =  {fiU)}-  l^ick  another  set  of  N  points  chosen  from  a  uniform  distribution 
in  the  domain  and  denote  this  by  =  {u(^)}  for  z  €  {0,  •  •  •  ,iV  —  1}.  Then  w’e 
have 


115^11?, 


=  E 


IEiIiexp(-i(fci/i(<i)  +  fc2/2(<i)))l 

1+k^  +  k^ 


(6.10) 


Here,  /i(^t)  and  f2{ti)  are  the  x  and  y  components  of  f{ti)  respectively.  The  mix- 
norm  for  Un  is  similarly  computed.  Note  that  exp  (— i  (fci/i  (<i) -h  A:2/2(^))) 
is  just  the  Fourier  component  of  the  S  distribution  given  by  the  samples  of  the 
trajectory  fa{x,y)  =  Eili  “  fi{U))S{y  -  f2{ti)). 
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Multiple  Target  Detection  using  Bayesian  Learning 


Sujit  Nair,  Konda  Reddy  Chevva,  Houman  Owhadi  and  Jerrold  Marsden 


Abstract 

The  need  to  develop  fast,  robusi  multiple  target  search  algorithms  has  generated  a  lot  of  interest  recently 
among  scientists  and  mathematicians.  In  this  paper,  we  develop  a  computationally  efficient  multiple  target  search 
.strategy  given  a  fixed  number  of  .search  vehicles  and  fixed  number  of  stationary  targets  in  a  region.  Two  different 
cases  depending  on  whether  the  number  of  targets  is  known  or  unknown  are  considered.  The  search  area  is 
divided  into  cells.  The  belief  map  is  updated  using  Bayes’  theorem  and  an  optimal  reassignment  (teleporting)  of 
vehicles  based  on  the  values  of  the  current  belief  map  is  adopted.  Exact  computation  of  the  belief  map  update 
is  expensive  and  often  an  approximation  is  needed.  In  this  paper,  we  show  that  the  Bayesian  update  can  be 
exactly  computed  in  an  efficient  manner  by  using  the  detection  history  in  each  cell  and  results  from  the  theory 
of  symmetric  polynomials. 


1.  INTRODUCTION 

Multi-target  detection  and  tracking  [1],  [2],  [3],  [4]  are  important  elements  of  a  surveillance  system.  In 
multiple  target  detection  and  tracking  problems,  one  is  interested  in  determining  the  number  as  well  as  the 
dynamics  of  targets.  Radar  and  sonar  based  tracking  of  objects  for  air  traffic  control  and  navigation  are  some 
of  the  applications  of  multi-target  detection  and  tracking.  In  order  to  successfully  detect  and  track  targets,  one 
needs  to  effectively  extract  useful  information  about  the  target's  state  from  observations. 

In  this  paper,  we  restrict  out  attention  to  detection  of  an  unknown  number  of  stationary  targets  using 
measurements  from  a  fixed  number  of  search  vehicles.  Though  we  mainly  focus  on  multi-target  detection, 
tracking  and  detection  are  closely  related  areas  with  significant  overlaps.  We,  therefore,  begin  with  a  brief  review 
of  the  techniques  for  multi-target  tracking  and  detection.  One  of  the  earliest  techniques  for  multi-target  tracking 
is  the  multiple  hypothesis  tracking  [I],  [2].  In  multiple  hypothesis  tracking,  one  associates  a  set  of  detections  of 
the  target  position  with  existing  tracks.  This  association  if  successful  leads  to  new  tracks.  Otherwise,  the  set  of 
detections  are  deemed  false  alarms.  Typically,  Kalman  filter  type  algorithms  are  used  to  update  the  existing  tracks 
after  association.  Multiple  hypothesis  tracking  suffers  from  some  problems.  For  instance,  some  information  is 
lost  when  the  detections  are  generated  from  raw  sensor  returns.  So  if  the  targets  rarely  produce  returns  above 
the  detection  threshold,  then  the  tracking  algorithm  fails  to  accurately  track  the  targets.  A  more  robust  solution 
to  multi-target  tracking  is  called  the  track-before-detect  [5]  where  raw  sensor  returns  are  available  to  the  tracker. 
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The  above  approaches  suffer  from  two  drawbacks.  First,  the  above  approaches  are  not  recursive  in  nature. 
Secondly,  if  the  target  motion  is  complicated,  then  the  above  approaches  have  difficulty  in  modeling  the  target 
motion.  Multi-target  detection  and  tracking  using  a  Bayesian  perspective  is  a  more  promising  and  robust 
approach.  This  is  the  approach  followed  in  this  paper.  Stone  and  coworkers  [3]  developed  a  mathematical 
theory  of  multi-target  detection  and  tracking  from  a  Bayesian  point  of  view.  Some  early  work  in  this  area  was 
done  by  Miller  and  coworkers(6],  Kastella  [7]  and  Mahler  [8].  In  the  Bayesian  approach,  the  multi-target  state 
is  a  markov  process.  The  Bayesian  approach  can  handle  complicated  target  dynamics.  It  also  provides  a  solution 
that  is  recursive  in  nature. 

The  problem  of  simultaneous  detection  and  tracking  of  multiple  targets  can  be  formulated  in  the  Bayesian 
framework.  The  problem  can  be  solved,  in  principle,  by  exact  computation  of  the  Joint  Multi-target  Probability 
Density  (JMPD)  [7],  [9]  that  accounts  for  the  uncertainty  about  the  number  of  targets  and  their  states.  The  JMPD 
is  a  high-dimensional  quantity  and  its  exact  computation  is  almost  impossible.  A  big  challenge  in  multi -target 
detection  and  tracking  research  is  to  develop  sophisticated  numerical  techniques  to  approximate  the  JMPD. 
Particle  filter  methods  that  provide  a  stochastic  grid  approximation  to  the  exact  solution  of  Bayesian  state 
estimation  have  been  proposed  to  approximate  the  JMPD  for  cases  involving  a  large  number  of  targets  moving 
in  two-dimensions  [10],  [11].  More  recently,  a  unified  approach  to  multi-target  detection  and  tracking  based  on 
recursive  approximation  of  the  JMPD  was  presented  in  [12]  where  an  efficient  particle  filtering  scheme  was 
proposed. 

Clearly,  an  important  research  theme  in  multi -target  detection  and  tracking  is  the  development  of  efficient 
computational  techniques  to  approximate  the  exact  solution  of  the  Bayesian  state  estimate.  In  this  paper,  we 
consider  the  problem  of  detecting  an  unknown  number  of  stationary  targets.  Since  we  only  deal  with  detection, 
association  is  not  a  concern.  Though  the  detection  problem  is  simpler  in  nature  than  the  tracking  problem,  we 
would  like  to  emphasize  that  the  exact  computation  of  the  Bayesian  update  becomes  increasing  challenging  as 
the  number  of  unknown  targets  and  the  grid  size  increases.  The  main  motivation  for  this  work  is  to  develop 
fast,  computationally  efficient  techniques  for  exact  computation  of  the  Bayesian  update. 

The  main  contributions  of  the  paper  are  as  follows.  We  have  demonstrated  optimal  Bayesian  updates  for 
detecting  unknown  number  of  targets  in  a  given  search  region.  The  belief  states  of  the  system  grow  expo¬ 
nentially  and  usually  cannot  be  solved  exactly.  However,  we  show  that  we  can  indeed  solve  the  problem  in  a 
computationally  efficient  way  by  using  the  detection  history  in  each  cell.  We  also  show  that  using  Newton’s 
identities  from  the  theory  of  symmetric  polynomials  [13]  helps  us  to  exactly  update  the  belief  map.  Newton’s 
identities  relate  two  different  ways  of  describing  the  roots  of  a  polynomial.  They  have  applications  in  invariant 
theory  and  combinatorics  and  have  connections  to  algebraic  geometry.  To  the  best  of  our  knowledge,  this  is 
the  first  time  that  Newton’s  identities  have  been  used  in  multiple  target  detection  problems. 

The  paper  is  organized  as  follows.  In  Sec.  11  we  review  the  well  known  Bayes’  theorem  for  calculating 
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conditional  probabilities.  In  Sec.  Ill,  we  formulate  the  search  problem  given  a  fixed  number  of  search  vehicles 
and  targets.  In  Sec.  IV,  we  propose  a  new  way  to  calculate  the  belief  map  in  a  computationally  efficient  manner. 
We  also  present  a  novel  application  of  Newton’s  identities  from  the  theory  of  symmetric  polynomials  to  exactly 
calculate  the  belief  map.  Simulation  results  are  presented  in  Sec.V. 


II.  Conditional  Probability  and  Bayes’  Theorem 


Conditional  probability  is  the  probability  of  some  event  given  the  occurrence  of  some  other  event.  Let 


(O,  F,  P)  be  a  probability  space  where  ft,  F  and  P  have  their  usual  meaning.  Let  A,  B^F  be  two  events 
with  P(B)  >  0.  The  conditional  probability  of  A  given  B  is  defined  as: 


P{A\B)  = 


{P{AnB) 

P(B) 


(1) 


where  P(/tnP)  is  the  joint  probability.  Similarly,  the  conditional  probability  of  B  given  A  is  given  as 


P(B\A)^ 


(P(/tnB) 

P(A) 


(2) 


Bayes’  theorem  for  conditional  probabilities  follows  from  equations  (I)  and  (2), 


P(A\B)  = 


P(B\A)P(A) 

P{B) 


(3) 


FYobabilities  P{A\B),  P{A),  P{B\A)  and  P(B)  are  usually  referred  to  as  posterior,  prior,  likelihood  and  marginal 


respectively.  More  generally,  if  {A,  }  is  a  partition  of  A.  then 


for  any  A,  in  the  partition. 


P{Ai\B)^ 


P(B\Ai)P(Ai) 

l,jP{B\Aj)P{Aj) 


(4) 


111.  Problem  FORMULATION 

The  main  problem  we  are  interested  in  is  the  following:  Given  a  search  area  with  a  fixed  number  of 
stationary  targets  and  search  vehicles,  design  computationally  efficient  strategies  for  detecting  all  the  targets. 
More  precisely,  consider  a  search  area  that  is  divided  into  n  cells  as  shown  schematically  in  Fig.  I.  Let  V  be  the 
total  number  of  search  vehicles.  We  will  consider  two  cases.  In  one  case,  the  number  of  targets  is  known  apriori. 
In  the  second  case,  the  number  of  targets  is  not  known.  However,  we  assume  that  there  is  an  upper  bound  T 
for  the  number  of  targets  in  the  region.  We  assume  that  the  targets  are  stationary.  We  adopt  a  simple  model  for 
the  vehicle  dynamics.  At  each  time  step,  the  vehicles  can  jump  to  any  other  cell  or  remain  in  their  current  cell. 
Each  cell  can  only  be  occupied  by  a  single  search  vehicle  and/or  a  single  target.  The  vehicles  detect  a  target 
with  probability  p^.  We  will  also  consider  false  alarms  with  probability  p/.  At  each  time  step,  the  vehicles 
send  their  detection  data,  i.e.,  detect  or  no  detect,  to  a  central  manager,  that  updates  the  belief  map  based  on 
the  vehicle  measurements.  The  belief  map  is  a  vector  of  numbers,  P(7)),  7=  1, n,  where  P{Tj)  denotes 
the  probability  that  the  target  is  in  cell  J.  The  vehicles  are  then  reassigned  to  new  cells  based  on  the  updated 
belief  map.  There  are  several  ways  to  reassign  the  vehicles  to  new  cells.  In  this  paper,  we  adopt  an  ideal  policy 
where  the  vehicles  are  reassigned  or  teleported  to  cells  that  correspond  to  the  maximum  values  of  the  belief 
map.  Though  this  reassignment  scheme  is  not  entirely  realistic,  it  provides  lower  bounds  for  the  detection  times. 
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#  search  vehicle  #  target 


Fig.  1.  A  schematic  of  the  search  grid  with  fixed  number  of  search  vehicles  and  targets 


In  order  lo  illustrate  the  application  of  Bayes’  theorem  for  the  search  problem,  consider  the  simple  case  of 
a  single  target  and  a  single  search  vehicle  in  a  grid  of  size  n.  The  essential  idea  is  to  update  the  belief  map  at 
each  lime  step  based  on  the  measurcmcnts  made  by  the  vehicles.  We  assume  that  the  initial  target  distribution 
is  uniform,  that  is,  P(7))^  =  1  /n  for  all  j.  Suppose  that  the  vehicle  is  in  cell  i  and  that  di  is  the  detection  data. 
That  is,  di  =  1  if  the  vehicle  detects  and  di  =  0  if  the  vehicle  fails  to  detect.  Then,  given  the  detection  data  di^ 
Bayes’  theorem  can  be  used  to  update  the  belief  map: 

P(Tj)' =P(Tj\di)  = 


Note  that 


P{di\Tj)P{Tjf 

P(d,) 

P(d.\Tj)P{Tj)° 
LUP{d,\Tk)P(Tkf  ■ 


P(di\Tj) 


Pd 

if 

di=\ 

& 

i  =  J 

Pf 

if 

di=\ 

& 

i^j 

1- 

Pd 

if  di 

=  0 

&  i  =  j 

1- 

Pf 

if  d, 

=  0 

&  iV  ;■ 

(7) 

(8) 
(9) 

(10) 


The  above  formula  can  be  easily  extended  lo  the  case  of  multiple  vehicles.  Before  we  do  that,  we  introduce 
some  notation.  stands  for  the  usual  conditional  probability  of  A  given  B.  .  .,7/^)  denotes  the 

probability  of  the  event  Ti^r\...r\Tij.  Here  it  is  assumed  that  the  indices  are  distinct.  By  D\D/,  we 

mean  the  detection  data  consisting  of  all  cells  except  the  cell.  By  P(A,, , . . .  yAi^  |B),  wc  mean  the  sum 
over  all  for  (J)  ways  of  choosing  indices  ii,...,/*  from  the  set 


One  observes  that  if  there  are  k  targets,  then  the  belief  map  consists  of  (J)  numbers  given  by  ^(7],,.. . ,7iJ. 
The  number  of  computations  required  to  update  the  belief  map  blows  up  exponentially  as  the  number  of  cells 
and  targets  increases.  For  example,  for  a  50  x  50  grid  with  5  targets,  the  belief  map  is  of  the  order  of  lO'^. 
This  is  a  very  important  consideration  if  a  particular  search  strategy  needs  to  be  implemented  in  real  time.  In 
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the  next  section,  we  describe  a  simple,  computationally  efficient  and  exact  method  to  update  the  belief  map. 
This  method  is  particularly  attractive  when  the  number  of  targets  and  grid  size  is  large. 

IV.  COMPUTATtONALLY  EmCtENT  BELtEF  MaP  UPDATE 
In  this  section,  we  propose  an  alternative  formulation  where  instead  of  having  a  belief  map  of  size  (J),  we 
construct  a  belief  map  of  size  n  by  considering  the  history  of  detections/no  detections  in  each  cell.  At  each 
instant  of  lime,  vehicles  make  measurements  about  targets  in  their  respective  cells  and  report  back  to  a  central 
manager  whether  they  have  detected  or  not  detected  a  target  in  their  respective  cells.  For  each  cell,  the  total 
number  of  deieaions  and  total  number  of  no  deteciions  is  stored.  Using  this  information,  at  each  time  step, 
the  belief  map  is  constructed  as  follows:  If  di  and  nij  are  the  total  number  of  detections  and  no  detections 
respectively  in  cell  i  up  to  that  time,  the  detection  data  can  be  written  as  the  following  2xn  matrix 

d\  ...  dfi 

D=  (II) 

mi  ...  m„ 

A.  Known  Number  of  Target  Case 

We  now  consider  the  case  when  the  number  of  targets  is  known  apriori  and  is  assumed  to  be  k.  For  the 
detection  data  £),  we  can  construct  the  belief  map  as  follows: 


P{Ti\D,U=k) 


P(D\T„U  =k)P(Ti\U  =k) 

P(D\U  =  k) 

E(^,_)P(D|7;,7-y . ,Tj,_„U  =  k) 

P{D\U=k) 

X  L  P{Tj . Tj,^,\Ti,U=k)P(T,\U=k) 

where  U  is  the  number  of  targets.  The  denominator  can  be  written  as 
P(D\U  =  k) 


=  LP(D|7) . Tj,,U  =  k)P(Tj„...Jj^\U=k)  . 

(1) 
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Assuming  the  initial  prior  is  uniform,  we  get 
P{Ti\D,U  =k) 


P{Tj . Tj,jTi,U  =k)P{Ti\U  =k) 

PiL . Tj,\U  =  k) 

. 

L^^,)PWTJ . rj,,U=k) 

. = 

^  L(r)PWPj . Tj,,U  =  k) 

L^^n^)P{D\Ti,Tj„...,Tj^_„U=k) 

^:)PWTj . Tj,,U=k) 

Therefore,  assuming  that  the  detections  are  independent  of  each  other,  we  can  write  the  above  equation  as 

P(Ti\D,U=k) 


l,,.y(D\Di\Tj, . Tj,„U  =  k) 

P{Di\1i)  . Tj„U=k) 

TTie  main  advantage  of  this  formulation  is  that  we  have  a  very  significant  reduction  in  memory  requirement 
as  we  are  only  storing  2n  numbers  at  each  stage  instead  of  (J).  We  will  later  show  how  to  exactly  compute 
the  numerator  and  denominator  in  the  above  equation  using  Newton's  identities  from  the  theory  of  symmetric 
polynomials  that  further  leads  to  computational  savings. 

B.  Unknown  number  of  Target  Case 

In  this  case»  we  consider  that  the  number  of  targets  is  unknown  apriori.  However,  we  assume  that  the 
maximum  number  of  targets  possible  is  f .  This  case  is  considerably  difficult  than  the  previous  case  where  the 
number  of  targets  were  known  apriori.  The  belief  map  update  proceeds  in  the  following  manner 

P{Ti\D) 

T 

=  Y,PUi\D,U=j)P(U=j\D) 

j=\ 

^  P(D\T,,  U  =  j)P{Ti\U  =  j)P(U  =  j\D) 

P(D\U=j) 

f  P(DiriM  =  mviic/  =  J}P(C/  =  j) 

h  P(D) 

j.  P(D\Ti.U  =  j)P(Ti\U  =  mu=j) 

h  LltP{D\U  =  l)P{U  =  l} 
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Assuming  uniform  prior  for  P{U  =  _/),  we  get 

P{Ti\D) 

j.  P{D\Ti,U  =  j)Pmu  =  J) 

£j.,P(D|7;-.f/  =  ;)P(7;|f/  =  ;) 

LLp(D\U=I) 

After  a  few  manipulations,  we  get 

P(7i|D) 

lu  ^o:,)  p(p>\p\'Ph 

. T,,,U  =  l)  ■ 

Now,  for  both  the  single  and  multiple  target  case,  we  need  to  compute 

P(D|7;-, , . . . ,  7]-^,  f/  =  ;)  =  n  PiDiTi,  = 

/=! 

where  D/  is  the  detection  data  corresponding  to  cell  /.  Define 

a,  =  P(D,|7;-, ,...,T,.,U  =  j)=(^'  +  pj ( I  - prf)"' 

if  /  €  . .,//}  and 

bi  =  P(Di\Ti, . Ti^,U  =  j)  =  (‘^'^^'"')p/'(l  -P/P 

Otherwise.  Therefore,  we  need  an  algorithm  to  efficiently  compute  terms  of  the  form 

’£P{D\Ti„...,TijM=j) 

(") 


=  'LYlP(P>l\Ph^  -Jl;^U=j)  (12) 

Q'-' 

(3) 

This  can  be  written  as 

J^P(D|7;-,,...,7-i,,f/=2)  =  PB  E  c/p-.c-V  (14) 

I  <M  <•••<*;-” 

where  Pb  =  b\ .  ..b„  and  c,  = 


As  one  can  see,  computing  P(71|D)  involves  a  larger  number  of  operations  and  can  considerably  slow  down 
the  belief  map  update.  We  are  faced  with  the  following  problem.  Given  n  numbers  ci,...Cn  and  an  integer  k. 
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compute  the  symmetric  polynomials 

Jk  =  E  (15) 

l<ij  <...<ii<n 

in  an  efficient  manner?  In  the  next  section,  we  briefly  review  the  theory  of  symmetric  polynomials  and  show 
how  the  symmetric  polynomials  can  be  expressed  in  terms  of  the  power  sums  using  Newton 's  identities.  This 
novel  application  of  Newton’s  identities  considerably  reduces  the  number  of  computations  needed  to  update  the 
belief  map. 

C.  Symmetric  Polynomials  and  Newton's  Identities 

A  symmetric  polynomial  on  n  variables  is  a  function  that  is  invariant  to  any  permutation  of  its 

variables.  That  is,  the  symmetric  polynomials  satisfy 

/(.Vl,  .V2  yn)=f(x\.  X2,  JTn),  (16) 

where  Vi  =Xxi  and  k  being  an  arbitrary  permutation  of  the  indices  I,2,...,n.  The  elementary  symmetric 
polynomials  X2,...,  x„)  are  given  by 

■'l(•r|,•r2 . -Tn)  =  Y. 

I  <i<n 

J2{X\y  X„)  =  Y, 

\<i<j<n 

MX],  X2,...,  X„)  =  Y 

\<i<j<k<n 


J„(X],  X2,...,  X„)  =  Yl 

\<i<n 

The  power  sum  Sp{x\y  X2,  ..,  is  defined  as 

n 

Sp(X],  -Tn)  =  Y^k 

*=1 

The  relation  between  the  symmetric  polynomials  y*  and  the  power  sums  Sp  is  given  by  Newton's  identities. 
The  first  few  identities  are 


5i 

(18) 

?-52) 

(19) 

f-3S,S2  +  2S3) 

(20) 

-  6S?S2  +  3S|  +  8S|  S3  -  6S4 ) 

(21) 
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In  general,  Ihe  symmetric  polynomials  can  be  computed  using  the  following  determinant 


Si 

1 

0 

0 

...  0 

jSi 

1 

0 

...  0 

iS3 

3S2 

5S1 

1 

...  0 

’  •  1 

(22) 

is* 

is*-i 

iS*-2 

1 

•  is, 

All  of  these  identities  can  be  computed  very  easily  using  any  commercial  symbolic  package.  The  main 
advantage  of  writing  it  in  this  form  is  that  computing  5,-  is  much  cheaper  as  it  involves  vector  processing.  For 
example,  if  n  —  100  and  it  =  4,  the  brute  force  implementation  using  four  for  loops  takes  around  25  seconds  on 
a  MacBook  Pro  laptop  whereas  using  the  Newton’s  identities,  it  takes  only  about  10  seconds!  This  clearly 
shows  that  one  can  get  computational  speed-up  of  orders  of  magnitude  by  using  Newton’s  identities. 

V.  StMULATioN  Results 

In  this  section,  we  provide  simulation  results  for  a  grid  of  size  15  x  15.  Note  that  the  state  space  for  the 
system  is  for  5  targets.  We  choose  5  target  at  locations  43,77,99,155,216  at  /  =0.  The  initial  prior 
is  assumed  to  be  uniform.  The  values  of  and  p/  are  chosen  to  be  0.9  and  0.1  respectively.  As  shown  in 
the  Figure  (2)  ,  the  belief  map  converges  to  the  expected  value  in  about  80  iterations.  The  optimal  teleponing 
scheme  gives  lower  bounds  on  the  detection  times  and  serves  as  a  baseline  against  which  other  strategies  can  be 
compared.  An  immediate  extension  of  this  strategy  is  local  optimal  teleponing  where  each  vehicle  is  moved  to  a 
neighboring  cell  with  the  maximum  belief  map  value.  This  is  ongoing  work.  Preliminary  results  suggest  that  the 
local  optimal  reassignment  strategy  performs  satisfactorily.  The  vehicle  dynamics  are  more  realistic  in  this  case. 

We  have  also  examined  the  effect  of  the  number  of  search  vehicles  and  pd  on  the  detection  times.  As 
expected,  as  the  number  of  search  vehicles  increases,  the  detection  time  decreases  with  the  number  of  search 
vehicles.  One  such  simulation  result  is  shown  in  Fig.  3.  However,  it  is  interesting  to  note  that  beyond  a  critical 
value  of  the  number  of  vehicles,  the  detection  time  does  not  change  much  with  the  number  of  vehicles.  Such  an 
observation  can  provide  guidelines  in  choosing  the  number  of  search  vehicles  for  a  given  search  mission.  Figure 
4  shows  the  variation  of  the  detection  times  with  p^.  Once  again,  as  expected  the  detection  limes  decreases 
almost  linearly  with  pd. 

VI.  Conclusions 

In  this  paper,  we  have  demonstrated  optimal  Bayesian  updates  for  detecting  unknown  number  of  targets  in 
a  region  that  is  divided  into  cells.  The  belief  states  of  the  system  grow  exponentially  and  cannot  be  solved 
exactly.  We  develop  a  different  formulation  and  show  that  we  can  indeed  solve  the  problem  exactly  using  results 
from  the  theory  of  symmetric  polynomials  and  using  the  detection  history.  As  future  work,  we  will  provide 
theoretical  estimates  of  the  detection  times  when  the  vehicles  are  optimally  teleported.  We  will  also  compare 
local  optimal  vehicle  reassignment  with  teleporting. 
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Iteration  #1 


Iteration  #20 


0.01 


0.005 


100  200 

Iteration  #40 


Iteration  #82 


0.5 


0  100  200 

Fig.  2.  Evolution  of  belief  map  for  a  15x15  grid.  The  cell  number  Is  plotted  on  the  x-axIs  and  the  belief  map  values  arc  plotted  on  the 
v-axis.  After  the  82nd  iteration,  targets  are  detected  at  location  43.77,99. 155.216. 
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1  Abstract 

Many  large  scale  systems  of  interest  (e,g.  power  systems,  biological  networks)  are  often  composed 
of  weakly  interacting  subsystems.  We  propose  an  iterative  scheme  that  exploits  such  weak  inter¬ 
connections  to  overcome  dimensionality  curse  associated  with  traditional  uncertainty  quantification 
methods  and  radically  accelerate  uncertainty  propagation  in  systems  with  large  number  of  uncertain 
parameters.  This  approach  relies  on  integrating  graph  theoretic  methods  and  waveform  relaxation 
with  uncertainty  quantification  techniques  like  probabilistic  collocation  and  polynomial  chaos.  We 
analyze  convergence  properties  of  this  scheme  and  illustrate  it  on  a  power  network. 


2  Introduction 

The  issue  of  management  of  uncertainty  for  robust  system  operation  Is  of  interest  in  a  large  family  of 
complex  networked  systems.  Such  systems  typically  involve  a  large  number  of  heterogeneous,  con¬ 
nected  components,  whose  dynamics  is  affected  by  possibly  an  equally  large  number  of  parameters. 
Uncertainty  Quantification  (UQ)  methods  provide  means  of  calculating  probability  distribution  of 
system  outputs,  given  probability  distribution  of  input  parameters.  One  of  the  most  commonly  used 
methods  is  Monte  Carlo  sampling  (MCS),  or  one  of  its  variants.  Although  MCS  is  straightforward  to 
apply  as  it  only  requires  repetitive  executions  of  deterministic  simulations  of  the  system,  typically  a 
large  number  of  such  executions  are  needed  as  the  solution  statistics  converge  relatively  slowly,  e.g., 
the  mean  value  typically  converges  as  0(1 /y/N)  where  N  is  the  number  of  realizations  [13].  Quasi 
Monte  Carlo  (QMC)  methods  on  the  other  hand  offer  better  convergence  for  moderate  number  of 
random  parameters,  the  rate  being  where  p  is  the  number  of  random  parameters  in  the 

system.  However,  while  MCS  suffer  from  poor  coverage  of  the  space  being  sampled,  QMC  methods 
often  lead  to  incorrect  density  of  the  sampled  points.  Recently,  Mezic  [11]  developed  a  new  scheme 
known  as  DSample,  which  exploits  ergodic  dynamics  to  generate  samples  which  do  not  suffer  form 
such  limitation:  the  sample  points  have  proper  coverage  and  correct  density  in  the  sample  space. 
Most  importantly  the  convergence  of  this  method  follows  the  fast  0{\/N)  scaling,  independent  of  p 
the  number  of  random  parameters. 

Generalized  polynomial  chaos  (gPC)  is  another  recently  developed  technique  which  belongs  to 
the  class  of  non-sampling  UQ  methods.  In  gPC,  stochastic  quantities  are  expressed  as  orthogonal 
polynomials  of  the  input  random  pareimeters,  and  different  types  of  orthogonal  polynomials  can  be 
chosen  to  achieve  better  convergence  (under  certain  circumstances,  exponential  convergence  can  also 
be  attained)  [12].  When  applied  to  differential  equations  with  random  inputs,  the  gPC  expansion 
is  typically  combined  with  Galerkin  projection,  such  that  the  resulting  set  of  equations  for  the 
expansion  coefficients  are  deterministic  and  can  be  solved  via  conventional  numerical  techniques. 
However,  stochastic  Galerkin  (SG)  procedure  can  be  challenging  when  the  governing  stochastic 
equations  take  a  complicated  form.  To  this  end,  high-order  probabilistic  collocation  method  (PCM) 
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has  been  developed  [14].  PCM  combines  the  advantages  of  both  Monte  Carlo  sampling  and  gPC- 
Galerkin  method.  The  implementation  of  a  PCM  algorithm  is  similar  to  that  of  MCS,  i.e.,  only 
repetitive  realizations  of  a  deterministic  solver  is  required;  and  by  choosing  a  set  of  sampling  points 
based  on  the  theory  of  multivariate  polynomial  interpolations,  it  retains  the  high  accuracy  and  fast 
convergence  of  gPC  expansion,  similar  to  SG.  In  higher  dimensions,  however,  the  ase  of  standard 
tensor  products  of  one-dimensional  quadrature  points  as  sampling  points  leads  to  an  exponential 
growth  of  the  number  of  points.  The  work  of  [14],  is  a  first  systematical  attempt  to  avoid  using  tensor 
product  constructions.  Instead  it  employs  the  so-called  sparse  grid  [5],  to  tackle  problems  with  large 
number  of  random  variables  more  efficiently.  In  addition,  to  deal  with  UQ  in  PDE  systems,  mutli 
element  formulation  of  gPC  and  PCM  have  also  been  recently  developed  [7,  2]. 

As  described  above,  while,  there  have  been  various  efforts  to  overcome  dimensionality  curse  asso 
ciated  with  UQ  methods,  none  of  such  extension  exploits  the  underlying  structure  and  dynamics  of 
the  networked  systems.  In  fact,  many  networks  of  interest  (e,g.  power  systems,  biological  networks), 
are  often  composed  of  weakly  interacting  subsystems.  As  a  result,  it  is  plausible  to  simplify  and 
accelerate  the  simulation,  analysis  and  uncertainty  propagation  in  such  systems  by  suitably  decom¬ 
posing  them.  For  instance,  authors  in  control  theory  studied  large-scale  interconnected  dynamical 
systems  using  graph  theoretic  decomposition  of  the  system.  These  studies,  however,  concentrated 
mostly  on  questions  of  stability  and  robustness  (see  e.g.  [15,  16]).  In  contrast  Mezic  et  al.  [10], 
introduced  a  framework  for  studying  more  general  asymptotic  behavior  and  uncertainty  propagation 
in  such  multicomponent  nonlinear  systems.  They  showed  that,  the  ase  of  graph  decomposition  in 
conjunction  with  Perron  Frobenius  operator  theory  can  greatly  simplify  the  invariant  measure  struc¬ 
ture  and  uncertainty  quantification,  for  a  particular  class  of  networks.  While  this  approach  exploits 
the  underlying  structure  of  the  system,  it  does  not  take  advantage  of  the  weakly  coupled  dynamics 
of  the  subsystems. 

In  this  paper,  we  propose  an  iterative  UQ  approarh  that  exploits  the  weak  interactions  among 
subsystems  in  a  networked  system  to  overcome  the  dimensionality  curse  associated  with  traditional 
UQ  methods,  and  radically  accelerate  uncertainty  propagation.  This  approach  relies  on  integrating 
graph  decomposition  techniques  and  waveform  relaxation  scheme,  with  probablistic  collocation  and 
generalized  polynomial  chaos.  Graph  decomposition  can  be  realized  by  spectral  graph  theoretic 
techniques  to  identify  weakly  interacting  subsystems.  Waveform  relaxation,  a  parallel izable  itera¬ 
tive  method,  on  the  other  hand,  exploits  this  decomposition  and  evolves  each  subsystem  forward  in 
time  independently  but  coupled  with  the  other  subsystems  through  their  solutions  from  the  previous 
iteration.  At  each  waveform  relaxation  iteration  we  propose  to  apply  PC  at  subsystem  level  and  use 
gPC  to  propagate  the  uncertainty  among  the  subsystems.  Since  UQ  methods  are  applied  to  rela¬ 
tively  simpler  subsystems  which  typically  involve  a  few  parameters,  this  renders  a  scalable  iterative 
approach  to  UQ  in  complex  networks.  In  an  alternative  approach  the  random  differential  equations 
can  be  transformed  to  a  deterministic  system,  by  employing  a  stochastic  Galerkin  projection.  Sub¬ 
sequently,  graph  decomposition  and  waveform  relaxation  can  be  applied  to  accelerate  simulation  of 
this  deterministic  system,  leading  to  uncertainty  quantification  in  the  original  system. 

This  paper  is  organized  in  six  sections.  In  section  3  we  set  up  the  mathematical  framework 
and  state  precisely  the  problem  of  uncertainty  quantification.  Section  4  deals  with  spectral  graph 
decomposition  while  waveform  relaxation  is  described  in  section  5.  We  give  an  overview  of  gPC 
and  PCM  methods,  in  the  section  3.  These  techniques  form  the  basic  ingredients  of  the  scalable 
approaches  to  UQ,  which  are  discussed  in  section  7.  We  propose  two  such  iterative  approaches:  the 
first  one  requires  access  to  equations  which  describe  the  underlying  dynamics  of  the  system,  while 
the  other  one  treats  the  system  as  a  black  box.  In  section  8  we  illustrate  this  iterative  procedure  on 
a  simplified  power  system  network  and  numerically  analyze  its  convergence  properties.  Finally,  in 
section  9  we  summarize  the  main  results  of  this  paper  and  lay  down  some  future  research  questions. 
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3  Uncertainty  Quantification  in  Networked  Systems 

Consider  a  nonlinear  system  described  by  as  system  of  random  differential  equation 

X  =  f(x,C,f),  (1) 


where,  f  =  ,fn)  6  R"  is  a  smooth  vector  field,  x  =  are  state 

variables  and  ^  is  p  dimensional  vector  of  uncertain  parameters  of  interests. 

The  solution  to  initial  value  problem  x(to)  =  xo  will  be  denoted  by  x(f;^),  where  for  brevity  we 
have  suppressed  the  dependence  of  solution  on  initial  time  to  and  initial  condition  xq.  The  Jacobian 
J  associated  with  system  (1)  is  given  by 


^(x,C,0 


dxj 


(2) 


and  describes  the  linearized  dynamics  of  the  system  about  the  solution  trajectory  x(i;^).  The 
average  value  of  Jacobian  along  the  solution  for  nominal  value  of  parameters  will  be  denoted  by 

J  0^^]*  (3) 

Let  us  also  define  a  set  of  quantities 

Z  =  ,Zd)  =  0(x)  =  (9i(x,--  -  ,9,i(x))),  (4) 


as  observables  or  quantities  of  interests.  The  goal  is  to  numerically  establish  the  effect  of  input 
uncertainty  of  ^  on  output  observables  z. 

In  what  follows  we  will  adopt  a  probabilistic  framework  and  model  C  =  (Ci,?2»  *  *  *  at?  a  p— 
variate  random  vector  with  independent  components  in  the  probability  space  whose  event 

space  is  il  and  is  equipped  with  a— algebra  A  and  probability  measure  V.  Without  loss  of  generality, 
we  would  assume  that  these  parameters  (Ci  ,^2?  ■  ■  ■  » Cp)  are  mutually  independent  of  each  other.  Let 
Wi  :  Vi  he  the  probability  density  of  the  random  variable  with  Vi  =  ?t(n)  C  R  being 

its  image.  Then, 

=  v^er  (5) 

i=l 

is  the  joint  probability  density  of  the  random  vector  ?  =  (Ci,  •  •  •  ,?p)  with  the  support, 

V 

r  =  n  r.  c  R'’.  (6) 

t=l 

Without  loss  of  generality  we  would  assume  that  Pj  =  [—1  l],i  =  1,  •  •  •  ,p.  Naturally,  the  solution 
for  system  (1)  and  the  observables  (4)  are  functions  of  same  set  of  of  random  variables  ?,  i.e 


x  =  x(«;0,  z  =  =  ^(x). 


(7) 


As  described  in  the  introduction,,  this  problem  of  uncertainty  quantification  in  large  systems  (n  »  1) 
with  large  number  of  uncertain  parameters  (p  1)  is  computationally  intensive.  Recently  developed 
UQ  methods  like  Dsample  [11]  and  sparse-grid  based  probabilistic  methods  [14]  can  handle  this 
complexity  to  some  extent.  These  methods  however  do  not  make  use  of  the  underlying  structure 
and  the  dynamics  of  the  system,  which  can  be  often  be  taken  to  an  advantage.  The  goal  of  this 
paper  is  to  develop  scalable  uncertainty  quantification  approaches  which  exploits  this  structure  and 
dynamics.  The  key  methodologies  for  accomplishing  this  are  the  spectral  graph  decomposition, 
waveform  relaxation,  and  gPC  and  PCM,  which  are  discussed  in  the  subsequent  sections. 


3 


85 


B.l.  SCALABLE  UNCERTAINTY  QUANTIFICATION  IN  COMPLEX  DYNAMICAL 
NETWORKS 


4  Graph  Decomposition 

The  problem  of  partitioning  the  system  of  equations  (1)  into  subsystems  based  on  how  they  interact 
or  are  coupled  to  each  other,  can  be  formulated  as  a  graph  decomposition  problem.  Given  the  set  of 
states  ,  •  •  •  jXn  and  some  notion  of  dependence  Wij  >  0,  f  =  1,  •  •  •  ,  n,  j  =  1,  *  ■  *  ^  n  between  pairs  of 
states,  a  graph  G  =  {V^E)  can  be  constructed.  The  vertices  v,  in  this  graph  represent  the  states  Xi 
and  two  vertices  are  connected  with  an  edge  of  weight  iCjj,  if  iCy  between  the  corresponding  states 
Xi  and  Xj  is  positive  (or  larger  than  a  certain  threshold). 

Different  choices  of  the  weight  matrix  W  =  [iCjj],  would  lead  to  different  decompositions  of  the 
graph.  For  instance,  Mezic  et  al.  [10],  proposed  a  horizontal-vertical  decomposition  (HVD)  of  the 
graph  G  based  on  the  structural  properties  encoded  in  the  Jacobian  2,  associated  with  the  system 
1.  Specifically,  in  this  case 

Wij  =  X(|  I),  (8) 

where,  1  is  an  indicator  function.  Note  \Vij  =  1  if  state  i  affects  state  j,  and  Wij  =  0  otherwise. 
The  HVD  of  the  graph  is  obtained  by  recursively  identifying  transient  and  recurrent  non-null  sets  of 
the  Markov  chain  corresponding  to  the  weight  matrix  IV,  with  weights  given  by  (8).  Vertically,  the 
system  is  decomposed  into  a  linear  series  of  subsystems,  where  the  subsystems  above  is  influenced 
by  the  subsystems  below  but  not  vice  versa.  So.  the  input  signal  propagates  unidirectionally  from 
the  bottom  to  the  top.  Horizontally,  each  subsystem  is  decomposed  into  independent  groups  with  no 
edges  connecting  different  groups.  So,  each  group  has  its  own  input  and  output  and  are  functioning 
independently.  Each  group  in  HVD  is  a  connected  component  of  the  graph  and  HVD  creates  a 
partial  ordering  on  the  set  of  connected  components. 

Another  plausible  partition  of  G  is  to  assign  the  nodes  into  different  components  such  that  nodes 
in  the  same  components  are  strongly  coupled  and  nodes  in  different  components  are  weakly  coupled 
to  each  other.  This  requires  a  notion  of  coupling  strength  between  nodes  or  states,  which  would 
depend  on  the  nature  of  the  problem;  for  our  case,  we  propase  to  use 

+  (9) 

which  measures  the  interdependence  of  states  Xi  and  Xj  on  each  other,  corresponding  to  the  linearized 
dynamics.  The  problem  of  decomposition  can  now  be  formulated  as  follows:  we  want  to  find  a 
partition  of  the  graph  G  with  edge  weights  (9)  such  that  the  edges  between  different  components 
have  a  very  low  weight  and  the  edges  within  a  components  have  high  weight.  The  main  tools  for 
accomplishing  this  decomposition  are  graph  Laplacian  matrices.  There  exists  a  whole  field  dedicated 
to  the  study  of  those  matrices,  called  spectral  graph  theory  [19].  Note  that  in  the  literature,  there  is 
no  unique  convention  as  to  which  matrix  is  exactly  called  the  graph  Laplacian  and  how  the  different 
matrices  are  denoted.  In  the  following  we  always  assume  that  G  is  an  undirected,  weighted  graph 
with  weight  matrix  W,  where  Wij  =  Wji  >  0.  The  iinnormalized  graph  Laplacian  matrix  is  defined 
as 

L=D-n\  (10) 

where,  D  is  the  diagonal  degree  matrix  whose  ith  diagonal  entry  di  =  Wij.  There  are  two  other 
matrices  which  are  called  normalized  graph  Laplacians  in  the  literature.  Both  matrices  are  closely 
related  to  each  other  and  are  defined  as 

=  I  -  D  (11) 

L„„  =  D-'L  =  I  -  D-HV.  (12) 

The  procedure  to  decompose  the  graph  using  graph  Laplacian,  is  summarized  below,  details  can  be 
found  in  [19].  Given  m,  the  number  of  components  or  subsystems, 

1  Construct  the  graph  G  =  {V,E)  by  the  procedure  described  above.  Let  W  be  its  weighted 
adjacency  matrix. 
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2  Compute  the  graph  Laplacian  L. 

3  Compute  the  first  m  eigenvectors  ,  •  •  •  yV„,  of  L. 

4  Let  V  6  be  the  matrix  containing  the  vectors  Vi,  •  '  as  columns. 

5  For  i  —  1,  •  •  •  n  let  Uj  G  be  the  vector  corresponding  to  the  i-th  row  of 

6  Cluster  the  points  u,, i  =  1,  •  •  •  ,  n  in  with  the  k-means  algorithm  into  clusters  Ci ,  •  •  •  ,  Cm- 

7  Output;  Clusters  Ai,---  Mm  with  Ai  —  {j|uj  G  C,}. 

Topically,  m  the  number  of  weakly  coupled  subsystems  (or  components)  in  the  system,  is  not  known 
apriori.  The  spectral  gap  in  the  spectrum  of  the  Laplacian  can  be  used  to  identify  m.  For  discussion 
on  other  methods,  see  (19).  We  shall  denote  by 

Ni  =  {j:3keAi  &  31  e  Ajy  St  Ski  >  0}^  t  =  (13) 

the  set  of  indices  of  the  components  (or  subsystems)  to  which  the  t— th  component  (or  subsystem) 
is  weakly  connected. 

In  practise  HVD  and  spectral  decomposition  can  be  combined:  the  connected  components  iden¬ 
tified  in  horizontal  layers  after  HVD  can  be  further  decomposed  into  weakly  interacting  subsystems 
by  using  graph  Laplacians,  as  described  above.  In  summary  the  graph  decomposition  partitions  the 
system  into  appropriate  subsystems,  allowing  the  application  of  waveform  relaxation,  an  iterative 
scheme  to  accelerate  simulation  of  a  system  composed  of  weakly  coupled  subsystems. 

5  Waveform  Relaxation 

In  this  section  we  describe  the  basic  mathematical  concept  of  the  Waveform  Relaxation  (WR) 
method.  We  consider  dynamical  systems  which  are  described  by  a  system  of  differential  equations 
of  the  form  (1).  For  purpose  of  discussion  here,  we  would  assume  that  the  parameter  values  ?  in  the 
system  (1)  are  fixed.  The  general  structure  of  a  WR  algorithm  for  analyzing  system  (1)  over  a  given 
time  interval  (0  T]  consists  of  two  major  processes,  namely  the  assignment  partitioning  process  and 
the  relaxation  process  (17,  18]. 

In  the  assignment-partitioning  process,  the  system  is  partitioned  into  m  disjoint  subsystems  of 
equations.  This  partition  can  be  accomplished,  for  instance  by  the  graph  decomposition  procedure, 
as  described  in  the  previous  section.  Without  loss  of  generality,  we  can  rewrite  Eq,  1  after  the 
assignment- partitioning  process  as: 

yi  =  Fi(yi,di(<),Ai,<) 
y2  =  F2(y2,d2(<),  A2,0 

ym  —  Fni(yni)  dj,j(i),  Anj,  f) 

where,  for  each  i  =  1,  •  •  ♦  ,  m, 

Fi  =  {/2:jeA,}, 

y*  G  is  the  subvector  of  x  assigned  to  the  i— th  partitioned  subsystem,  i.e 

y>(<)  =  --j  e  Ai}, 

Ai(0  =  {?j:ieAj}, 

and 

di(0  =  {yj,(0 


(14) 

(15) 

(16) 

(17) 

(18) 
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are  decoupling  vectors.  It  is  clear  that  if  the  vectors  di{t)  are  treated  as  the  input  variables  of 
the  system  described  by  Ex}.  14,  then  the  system  can  be  easily  solved  by  solving  m  independent 
subsystems  associated  with  Fi,  •  •  •  ,Fm  respectively. 

The  relaxation  process  is  an  iterative  process,  w'hich  starts  with  an  initial  guess  of  the  waveform 
solution  of  the  original  dynamical  equations  (14)  in  order  to  initialize  the  approximate  waveforms  of 
the  decoupling  vectors.  During  each  iteration,  each  decomposed  subsystem  is  solved  for  its  assigned 
variables  for  t  €  [O  T]  by  using  the  approximate  waveform  of  its  decoupling  vector.  Two  most 
commonly  used  types  of  relaxation  are:  Gauss-Seidel(GS)  relaxation  and  the  Gauss* Jacobi  (GJ) 
relaxation.  For  the  GS  relaxation,  the  waveform  solution  obtained  by  solving  one  decomposed 
subsystem  is  immediately  used  to  update  the  approximate  waveforms  of  the  decoupling  vectors  of 
the  other  subsystems.  For  the  GJ  relaxation,  all  waveforms  of  the  decoupling  vectors  are  updated  at 
the  beginning  of  the  next  iteration.  The  relaxation  process  is  carried  out  repeatedly  until  satisfactory 
convergence  is  achieved. 

Let  the  superscript  index  /  denote  the  WR  iteration  count.  Then  the  general  structure  of  a  W’R 
algorithm  can  be  formally  described  as  follows: 

•  Step  0  (Assignment-partitioning  process):  Partition  (1)  into  m  subsystems  of  equations  as 
given  by  (14). 

•  Step  1:  (Initialization  of  the  relaxation  process);  Set  /  =  1  and  guess  an  initial  waveform 
{y?(0  :  f  e  [0  T]}  such  that  y?(0)  =  yoj. 

•  Step  2  (Analyzing  the  decomposed  system  at  the  I-th  WR  iteration):  For  each  i  =  1,  -  ••  ,7n, 
set 

d[(0  =  {yj,  *(0  :  ji  e  Ni}  (19) 

for  the  GJ  relaxation,  and  solve  for  {y^(0  :  <  €  [0  T]}  from 

yf  =  (20) 

with  initial  condition  y[(0)  =  y^o. 

•  Step  3  (Iteration)  Set  /  =  /  -h  1  and  go  to  step  2. 

Remarks 

1  A  simple  guess  for  {y^  (0}  y?(0  =  Xi(O)  for  all  i  G  [0  T]. 

2  In  the  actual  implementation,  the  relaxation  iteration  will  stop  when  the  difference  between  the 
waveforms  {y'(t)  =  (yf(0,---  >ym(0)  =  <  e  [O.T]}  and  {y'~'(t)  =  (y{“*(t),---  .y/,r*(0)  = 
t  G  [0  T]},i  .e.  supt^io  ||y/(0  —  y/_/(f)||,  is  sufficiently  small, 

3  In  analogy  to  the  classical  relaxation  methods  for  solving  linear  or  nonlinear  algebrsdc  equa¬ 
tions,  it  is  possible  to  modify  a  WR  algorithm  by  using  a  relaxation  parameter  cj  €  (0,2).  By 
introducing  ui,  the  iteration  equation  (20)  gets  modified  to  yield 

y[  =  F,(yf,df(t),A.,t),  (21) 

where, 

y.'=y.-“'+w(yf-y[-‘).  (22) 

Note  the  following  two  important  characteristics  of  the  WR  Algorithm: 

1  The  analysis  of  the  original  system  is  decomposed  into  the  independent  analysis  of  m  subsys¬ 
tems. 

2  The  relaxation  process  is  carried  out  on  the  entire  waveforms,  i.e.  during  each  iteration  each 
subsystem  is  individually  analyzed  for  the  entire  given  time  interv'al  [0,T]. 

The  conditions  that  guarantee  the  convergence  of  WR  method  have  been  analyzed  in  detail  in  [17]. 

6 


88 


B.l.  SCALABLE  UNCERTAINTY  QUANTIFICATION  IN  COMPLEX  DYNAMICAL 
NETWORKS 


6  Uncertainty  Quantification  Methods 

As  discussed  in  introduction,  UQ  methods  can  be  classified  as  sampling  and  non-sampling  based. 
In  this  section,  we  describe  two  such  interrelated  approaches:  generalized  polynomial  chaos  (gPC) 
and  probablistic  collocation  method  (PCM). 

6,1  Generalized  Polynomial  Chaos 

In  the  finite  dimensional  random  space  T  defined  in  (6),  the  gPC  expansion  seeks  to  approximate 
a  random  process  via  orthogonal  polynomials  of  random  variables.  Let  us  define  one-dimensional 
orthogonal  polynomial  spaces 

i-r''’''*  =  {u  :  Pfc ->  R  :  v  e  span{V',(Cfc)}fi„},  k=l,---,p,  (23) 

where,  denotes  the  polynomial  basis  from  the  so  called  Wiener-Askey  polynomial  chaos  [12]. 

According  to  the  Cameron- Mart  in  theorem  [21],  the  Wiener-Askey  polynomial  chaos  expansion  can 
approximate  and  describe  all  stochastic  processes  with  finite  second-order  moment,  which  is  satisfied 
for  most  physical  systems.  The  Askey  scheme  of  polynomials  contains  various  classes  of  orthogonal 
polynomials  and  with  their  associated  weighting  functions  which  coincide  with  probability  density 
function  of  different  distributions.  For  example,  uniform  distributions  are  associated  with  Legendre 
polynomials,  and  Gaussian  distributions  are  associated  with  Hermite  polynomials.  An  important 
property  of  the  Wiener-Askey  polynomial  chaos  is  orthogonality: 

'Pi{^k)^j(^k)wk(^k)d^k=Sij,  (24) 


where,  Sij  is  the  Kronecker  delta. 

The  corresponding  P- variate  orthogonal  polynomial  space  in  F  is  defined  as 

\V^  =  (g)  w*-^-  (25) 

|d|<P 

where  the  tensor  product  is  over  all  possible  combinations  of  the  multi-index  d  =  (dj,  d2>  *  *  *  ^ 

satisfying  |d|  =  di  <  P.  Thus  is  the  space  of  N-variate  orthonormal  polynomials  of 
total  degree  at  most  P,  and  its  basis  functions  satisfy 

^  'I'i(C)'I'j(C)w(C)rf4  =  =  Sij,  (26) 

for  all  1  <  i,j  <  dim(Wjf)  =  M  =  where  S  is  the  expectation  operator. 

The  major  advantage  of  applying  the  gPC  is  that  a  random  differential  equation  can  be  trans¬ 
formed  into  a  system  of  deterministic  equations.  A  typical  approach  is  to  employ  a  stochastic 
Galerkin  projection,  in  which  all  the  state  variables  xi  ,0:2,  •  •  ■  ,  Xn  are  expanded  in  polynomial  chaos 
basis  with  corresponding  modal  coefficients  (a{'(i)),  as 

M 

xr(«,C)  =  ^a?(0'I'.(C),  (27) 

t=l 

Substituting,  these  expansions  in  Ekj.  (1),  and  using  the  orthogonality  property  of  polynomial  chaos 
(25),  we  obtain 

aJ  =  ^A(xf(<,0,---,x,^(<.0,C,«)'I'j(0w(C)C  k=l,--,n,  (28) 
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a  set  of  deterministic  modal  ODEis,  with  initial  conditions 

a)(0)  =  ^xt(0,O^j(Ow(^K,  j  =  (29) 

This  system  can  be  solved  with  any  numerical  method  dealing  with  initial- value  problems,  e.g.,  the 
Runge-Kutta  method.  Similarly,  the  observable  can  be  expanded  in  gPC  basis,  as 

A/ 

=  k=l, ■■■,(!.  (30) 

t^i 

where, 

6)  =  ^^*(C)'I’>(Ow(OdC  =  ^P*(x(t,0)'I’j(Ow(Orf?,  k=  I, ■■■,(!,  j  =  l,---,A/.  (31) 

Hence,  once  the  solution  to  the  system  (28)  has  been  obtained,  the  coefficients  bj  can  be  approxi¬ 
mated  as 

bj  ~  k=\,  ■■■,(!,  (32) 

Such  a  Galerkin  procedure  has  been  used  extensively  in  the  literature.  However,  when  (1)  takes  a 
complicated  form,  the  derivation  of  Galerkin  projection  in  (28),  and  subsequently  the  gPC  approx¬ 
imation  of  the  observable  in  (32),  can  become  highly  non- trivial,  if  not  impossible.  To  circumvent 
this  difficulty,  probablistic  collocation  method  has  been  developed. 


6.2  Probabilistic  Collocation  Method 


The  collocation  method  is  an  alternative  approach  to  solve  stochastic  random  processes  with  the 
gPC.  Instead  of  projecting  each  state  variable  onto  the  polynomial  chaos  basis,  the  collocation 
approach  evaluates  the  integrals  of  form  (31)  by  evaluating  integrand  at  the  roots  of  the  appropriate 
basis  polynomials.  Two  underlying  concepts  for  the  PCM  are  the  orthogonal  polynomial  and  their 
associated  quadrature  rule.  Given  a  probability  density  fimction  it;(0  (l^t  p  =  l,d  =  1  for  now), 
the  PCM  based  on  Gauss  quadrature  rule,  approximates  an  integral  of  a  function  g  with  respect  to 
density  tc(C),  as  follows 


/: 


g(OMOd?  ~  ^  giriWi, 

necf 


(33) 


where, 

cl  =  [Vk  :  tp^+iivk)  =  0,/r  =  I,---  ,9+1},  (34) 

is  the  set  of  Gauss  collocation  points  with  xpi  being  the  orthogonal  polynomials  corresponding  to  the 
probability  distribution  ti;(0)  ^  described  in  previous  section.  The  weights  IPj  are  given  by 


vv,=  /‘r4 

7-1  (y-Ci 


A-(y) 


WpiXi 


■w(y)dy. 


(35) 


The  Gauss  quadrature  formula  which  is  a  well-known  numerical  integration  technique,  yields  an 
exact  integration  value  for  any  function  in  a  polynomial  form  of  order  less  than  or  equal  to  2(7  -}- 1. 
Other  quadrature  rules  can  be  also  be  u.sed,  some  of  which  have  nested  quadrature  points.  One  such 
rule  known  as  Clenshaw  Curtis  (CC)  quadrature  is  described  in  the  Appendix. 

For  higher  dimensional  discussion  (i.e.  for  p  >  1),  we  would  denote  ID  quadrature  rule  (Gauss 
or  CC)  along  each  random  dimension  as 


W,*  [y)  =  X]  ^^'iik9(ri,k), 


*=i 


7  =  I,---  ,P. 


(36) 
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where,  Ij  is  the  accuracy  level  of  quadrature  formula,  and  mi^  is  the  number  of  quadrature  points 
corresponding  to  that  accuracy  level.  Building  on  the  ID  quadrature  formula,  the  full  grid  PCM 
leads  to  following  cubature  rule, 

,Cp)wm  (37) 

mij  mi2  ^ip 

=  X]  ^  ("  'ui  "  fjjj  •  •  •  "  inJn )s(nu, ,  ■  ■  ■  ,  rfpjp)  (39) 

Jl=lj2  =  l  Jp=l 


To  compute  ,/p,p)  we  need  to  evaluate  the  function  on  the  full  collocation  grid  C(l,p) 

(where,  1  =  (/j,  •  ■  ■  ,/p))  which  is  given  by  tensor  product  of  ID  grids 

C(1,P)  =  C,‘  x-.-xC^,  (40) 

with  a  total  number  of  collocation  points  being  Q  =  h’  framework,  therefore,  for  any 

t,  the  approximations  to  the  model  coefficients  aj  (see  Eq.  27)  and  6^  (see  Eq.  30)  can  be  obtained 
as 


mi,  tniy  m/p 

a*(0=  f  Y.  Y  ■■■  (41) 

jl=lh=l  jp=i 

and 

/mj,  miy  mip 

gUx{t,0)'i'AO^{ad^^Y  Y  ■■■  -  ,r,^,J). 

jp=i 

(42) 

Note  to  compute  summations  arising  in  above  equations  (41,42),  the  solution  x(/,r/ij, ,  •  ■  ■  ,n^jp) 
of  the  system  (1)  is  required  for  each  sampling  point  (n,ji,---  iHpjp)  in  the  full  collocation  grid 
C(l,p).  Thus,  simplicity  of  collocation  framework  only  requires  repeated  runs  of  deterministic  solvers 
(without  explicitly  requiem  access  to  the  system  equations),  resulting  in  a  faster  algorithm  than  gPC. 

If  we  choose  the  same  order  of  sampling  points  in  each  dimension,  i.e.  li  =  =  /,,  =  /, 

the  total  number  of  points  Is  Q  =  F.  Hence,  the  computational  cost  increases  rather  steeply 
with  the  number  of  uncertain  parameters  p.  This  problem  can  be  circumvented  by  using  different 
cubature  rule.  One  such  example  is  Smolyak  rule  which  requires  significantly  smaller  number  of 
points  while  maintaining  the  same  accuracy  as  full  grid  PCM.  Smolyak ’s  algorithm  is  a  method 
first  developed  to  handle  high  dimensional  quadrature  [4]  and  later  extended  to  accomplish  high 
dimensional  interpolation  [6].  Its  basic  idea  is  to  use  the  solution  to  several  low-dimensional  problems 
to  span  the  space  and  then  linearly  combine  these  to  yield  the  solution  to  higher  dimensional  problem 
[5].  Grid  generated  using  this  rule  is  known  as  sparse  grid]  for  further  discussion,  the  reader  is 
referred  to  the  Appendix. 

7  Scalable  Uncertainty  Quantification  Approach 

In  this  section  we  discuss  how  generalized  polynomial  chaos  and  probabilistic  collocation  method  can 
be  integrated  with  graph  decomposition  and  waveform  relaxation  scheme,  leading  to  scalable  UQ 
approaches.  ■  We  describe  two  such  iterative  approaches:  the  first  one  requires  access  to  the  equations 
which  describe  the  underlying  d>Tiamics  of  the  system,  while  the  other  one  treats  the  system  as  a 
black  box. 
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7.1  Equation  Based  Approach 

The  procedure  of  transforming  a  random  differential  equation  to  a  deterministic  system,  by  employ¬ 
ing  gPC  and  stochastic  Galerkin  projection,  was  outlined  in  section  6.1.  The  deterministic  set  of 
ode’s  s*o  obtained  describe  the  evolution  of  modal  coeflicients  (see  section  6.1  for  details)  and  can 
be  compactly  written  as 

a  =  H(a,0,  (43) 

with  the  initial  condition  a(0)  (see  Eq.  29  as  well),  where, 

a  =  {a\,a'2,---  (44) 

H  =  (45) 

and 

-  M  M 

i-l  i=l 

(46) 

Note  that  order  of  the  deterministic  system  (43)  is  nA/,  where  M  =  and  is  significantly  greater 

than  the  order  n  of  the  original  random  system  (1).  However,  by  applying  graph  decomposition  and 
waveform  relaxation  to  the  deterministic  system  (43),  we  can  accelerate  the  computation  of  solution 
to  the  initial  value  problem  stated  above.  The  main  assumption  here  is  that  if  the  system  (1  is 
composed  of  weakly  interacting  subsystems,  so  would  be  the  transformed  system  (43).  In  summary, 
the  three  step  approach  to  UQ  is: 

•  Step  I:  Apply  gPC  and  stochastic  Galerkin  projection  to  system  (1)  to  obtain  a  deterministic 
system  (43). 

•  Step  II:  Apply  graph  decomposition  (see  section  4  for  details)  to  identify  weakly  interacting 
subsystems  in  the  system  (43). 

•  Step  III:  Apply  waveform  relaxation  (see  section  5  for  details)  to  the  decomposed  system 
obtained  in  Step  II, 

After  these  steps,  one  can  evaluate  the  effect  of  uncertainty  on  observables  as  described  in  section 
6.1.  One  of  the  major  limitation  of  the  above  approach  Is  that,  in  order  to  accomplish  step  1  we  need 
access  to  the  equations  (1),  which  may  not  be  readily  available  for  large  multicomponent  systems. 
Moreover,  even  when  equations  are  available,  the  Galerkin  projection  step  may  be  very  tedious,  if 
not  impossible.  We  describe  an  alternative  approach  in  the  next  section,  which  treats  the  system 
more  or  less  as  a  black  box. 

7.2  Probabilistic  Waveform  Relaxation  Based  Approach 

In  section  5  we  described  a  deterministic  waveform  relaxation,  in  which  the  decoupling  vectors 
are  deterministic  function  of  times.  By  incorporating  UQ  methods,  we  would  extend  WR  to  the 
probabilistic  setting  we  are  dealing  with.  Specifically,  at  each  waveform  relaxation  iteration  we 
propose  to  apply  PCM  at  subsystem  level  and  use  gPC  to  propagate  the  uncertainty  among  the 
subsystems.  Since  UQ  methods  are  applied  to  relatively  simpler  subsystems  which  typically  involve 
a  few  parameters,  this  renders  a  scalable  iterative  approach  to  UQ  in  large  networks.  Moreover,  like 
in  PCM  this  method  does  not  require  access  to  the  equations  describing  the  system  dynamics. 
Consider  the  t— th  subsystem  of  the  system  (14),  written  as  system  of  1st  order  ODE’s 

y\  = 

=  ^2(y>.di(0-An<) 

yi  =  F;(yndi(<),A.,0  (47) 
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where,  yj  =  (yj,  y2»  *  “  » 2/!v,)  Nf,  =  1^4*1  (where,  |  ■  |  denotes  the  cardinality  of  the  set)  and  di{t) 
is  the  decoupling  vector  (18).  Note  that  the  i— th  subsystem  is  directly  affected  by  the  parameters 
Ai  and  indirectly  by  other  parameters  through  the  decoupling  vector.  We  shall  denote  by  E*  C 
the  set  of  all  parameters  which  directly  or  indirectly  affect  the  i— th  subsystem. 
Clearly,  E*  D  At  and 

K  =  r\Ai  (48) 

will  be  the  set  of  parameters  which  indirectly  affect  the  i-th  subsystem.  Also  we  shall  denote  by 

Aj,  =  U  Aj.  (49) 

jeAT. 

Analogous  to  the  P- variate  orthogonal  polynomial  space  introduced  in  section  6.1,  we  consider  a 
P- variate  space  formed  over  the  random  parameter  set  EC  •  >Cp} 

U'l’ =  (g)  IV'*'-''*,  (50) 

|d|<P 

such  that  |d|  =  di+d2-\ - d^  <  P,  where  N  =  |E|.  Following  this  notation  for  the  t— th  subsystem, 

we  shall  denote  the  polynomials  belonging  to  VV^£,  by  =  l,*-*A/i,  where  A/j  =  and 

Ni  —  |Ej|.  In  this  polynomial  space  we  seek  polynomial  chaos  expansions  of  the  form, 


A/. 

yr(t,EO  =  X^a'*(0'I''(S'),  (51) 

j  =  l 

where,  the  superscript  i  in  the  above  variables,  denote  that  the  all  variables  correspond  to  the  t— th 
system.  We  shall  denote  by  yf(i,E*)  =  (yl^,  ^2^,  •  *  *  iy5v])  vector  of  P— variate  expansion  for 
i— th  subsystem.  With  such  an  expansion,  we  can  rewrite  each  equation  in  the  subsystem  (47)  as 

yi  =  f’*(yi>yi(^(i>s;.),A,-,o,  k  =  (52) 

where,  we  have  expressed  the  decoupling  vector  as  yidt,  E*)  =  (yj^  ((,&'),  ■  ■■  ,  yj’^  (t,  )),  with 

>  e  M  and  AT,.  =  |A/;|. 

The  full  collocation  grid  C(l,ni)  for  the  i— th  subsystem  will  be  represented  as 


C(l,nt nj*)  =  C(o,n,)  X  C(m,  rif),  (53) 

where,  1  =  (o,  m), 

C(o,  n.)  =  Co.  X  •  ■  •  X  C^„_ ,  (54) 

is  the  collocation  grid  corresponding  to  parameters  A,,  with  rij  =  |A{|.  o  =  (oi,  •  •  •  ,o„, ),  and 

C(m,0  =  C*.^  (55) 

is  the  collocation  grid  corresponding  to  parameters  EJ^,  with  =  |E*|  and  m  =  (mi,-*  -  ,m,j«). 
Since,  the  behavior  of  i— th  subsystem  weakly  depends  on  the  parameters  EJ,  we  can  take, 

max  rrii  -C  max  Ok.  (56) 

With  this  framework,  we  are  ready  to  outline  the  second  UQ  approach: 

•  Step  I:  Apply  graph  decomposition  (see  section  4  for  details)  to  identify  weakly  interacting 
subsystems  in  the  system  (1). 

•  Step  II:  Apply  probabilistic  waveform  relaxation,  which  involves  following  sub  steps. 
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—  Step  0  (Assignment-partitioning  process):  Partition  (1)  into  m  subsystems  (obtained  in 
Step  I)  loading  to  system  of  equations  given  by  (14).  Obtain,  Aj,  Af  E*  and  E*  for  each 
subsystem,  t  =  1,  • » •  ,  m. 

—  Step  1:  (Initialization  of  the  relaxation  process  with  no  coupling  effect  incorporated):  Set 
/  =  1  and  for  each  i  =  1,  ■  ■  •  ,m,  guess  an  initial  waveform  {y®(0  :  i  €  [O  T]}  such  that 
y®(0)  =  yoj,  so  that,  the  decoupling  vector  becomes 

ylit)  =  (yj,(0),  •  •  ■  ,y}s„  (0)),  jk  G  Ni,  N„  =  (57) 

and  solve  for  {y,-(^A*)  :  t  G  [0  T]}  from 

y.' =F-(y‘,y‘.(0,A„0,  (58) 

with  an  initial  condition  yi(0)  =  y?(0)  on  a  collocation  grid  C(o,ni).  Compute  the  gPC 
expansion  over  P- variate  polynomial  space  ,  leading  to 

A/. 

yr(t,A‘)  =  y]a‘*(0'I'‘(A‘),  (59) 

J  =  1 

for  A:  =  1,  •  *  ■  ,  Mi.  From  now  on  we  shall  denote  the  solution  vector  of  the  t— the  subsys¬ 
tem  at  /— th  iteration  by  yf^  —  ■  ■  ■  ,yjv]^). 

—  Step  2:  (Initialization  of  the  relaxation  process,  incorporating  first  level  of  coupling  effect): 
Set  1  =  2  and  for  each  i  =  1,  •  •  •  ,  m,  set 

yUt,  K)  =  (yj;' {t,  AA ), . . . ,  {t,  )),  j*  e  M,  N,,  =  |M|,  (60) 

for  the  GJ  relaxation,  and  solve  for  {y?(f,E*)  :  t  G  [0  T]}  from 

y?  =  F‘(y?,yL(<-A‘),Af,0-  (6i) 

with  an  initial  condition  y?(0)  =  y?(0),  over  a  collocation  grid  C(l,ni  -I-  nf).  From  this 
obtain  the  P— variate  expansions  over  the  polynomial  space  ,  so  that 

A/, 

=  (62) 


-  Step  3  (Analyzing  the  decomposed  system  at  the  I-th  WR  iteration):  For  each  t  = 
1,  ■  ■  •  ,  m,  set 


y/,(t,E-)  =  (y,^'^-'>(<,EA),...,y^ 


Jn„ 


(t,5:^^”)), 


iVn  =  ^A/;^  (63) 


for  the  GJ  relaxation,  and  solve  for  {y^(i,E‘)  :  f  G  (0  T]}  from 

y[  =  F'(y/,y[.(<,s'),Ai,o,  (64) 

with  initial  condition  y((0)  =  yi(0)  over  a  collocation  grid  C(l,ni  -I-  nj').  Obtain  the 
expansions, 

Ar. 

yr(i,A‘)  =  ^aj*(0'I'*(A’),  (65) 

—  Step  4  (Iteration)  Set  /  =  /  -f-  1  and  go  to  step  3. 
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Note  that  in  above  approach,  PCM  is  applied  at  subsystem  level  with  the  collocation  grid  C(l,  rii  -H 
nj)  (where,  (1  =  (o,m))with  o  =  (oi,***  ,o„, )  and  m  =  (mi,--  -  being  sparse  for  the 

parameters  which  affect  that  subsystem  indirectly  (see  condition  56),  The  table  below,  shows  how 
much  computational  savings  (see  last  column)  can  be  obtained  by  using  above  framework,  instead 
of  full  collocation  over  the  entire  parameter  space.  In  the  table,  m  be  the  number  of  subsystem, 
Pi  will  be  number  of  parameters  occurring  in  the  i— th  subsystem  and  /  be  the  order  of  accuracy 
of  collocation  along  each  dimension  for  the  collocation  grid  over  the  entire  parameter  space  with 
p  =  parameters.  Lets  assume  for  simplicity  of  analysis,  that  o,  =  =  I,  -  -  ,nj  and 

TTii  =  Icyi  =  !,•••  ,nj  and  Ir  <  1^.  /max  denote  the  maximum  number  of  waveform  relaxation 
iterations. 


Subsystems  &:  parameters 

Collocation  parameters 

Full  Collocation  (Up) 

Iterative  (7^/) 

m  =  2, Pi  =  5,1  =  1,2 

II 

II 

11 

to 

9,765,625 

2,006,251 

5 

m  =  3, Pi  =  5,i  =  I, ,  2,3 

/  =  5,  /,  =  5,  l„,  =  2 

3.05I8e-|-I0 

96,009,376 

300 

In  the  table  above  entries  in  third  column  are  computed  using  formula  Up  =  which  denotes  the 
number  of  deterministic  runs  of  the  complete  system  (1).  Similarly,  the  fourth  column  is  obtained 
using 

VI  TTl 

7^,  =  l  +  ^e  +  /^,ax(^e(g)/';0-  (66) 

assuming  /max  =  10,  and  measures  the  total  number  of  deterministic  runs  of  the  subsystems  in¬ 
volved.  Clearly,  advantage  of  the  iterative  approach  becomes  evident  as  the  number  of  subsystems 
and  parameters  in  the  system  increase.  Also  note  that  this  approach  is  parallelizable,  and  hence 
highly  scalable. 

7.3  Some  Remarks  on  two  algorithms: 

1  There  are  number  of  parameters  that  need  to  properly  identified  to 

2  Analytical  conditions  under  which  these  algorithms  will  converge,  is  not  known  and  need  to  be 
established.  These  conditions  would  provide  an  We  perform  some  experiments  to  numerically 
study  the  convergence  behavior  of  probabilistic  WR. 

3  Note  that  in  both  algorithm  graph  decomposition  is  applied  to  identify  weakly  interacting 
system,  before  waveform  relaxation  is  initiated.  It  is  assumed  that  this  decomposition  remains 
valid  as  the  system  evolves  during  relaxation  process.  However,  the 

4  The  full  grid  collocation  used  in  above  algorithm  can  be  replaced  by  sparse  grid  collocation 
ill  a  straightforward  manner.  With  this,  additional  computational  gain  can  be  attained  as 
sparse  grid  methods  are  computationally  efficient  comp2ired  to  the  full  grid  (see  section  ??  and 
Appendix  A  for  further  details). 

8  Example  Problems 

In  this  section  we  illustrate  the  iterative  procedure  developed  in  previous  section  on  a  simplified 
power  system  network  and  numerically  analyze  its  convergence  properties. 
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8.1  Stability  Problem 

In  order  to  illustrate  the  iterative  algorithm  proposed  in  section  7.2  and  study  its  convergence 
behavior,  we  first  consider  a  simple  system,  with  two  states  (a;i,2:2), 


xi  =  axj -\-cxl —vi,  (67) 

X2  =  cxj  -I-  60:2  -  t»2»  (68) 


where,  a,6,c,ui,U2  are  the  parameters.  Here  the  parameter  c  determines  the  coupling  strength 
between  two  subsystems  described  by  the  two  equations.  It  would  be  assumed  that  c, ?;i,t;2  are 
deterministic  parameters,  while  a,  b  are  uncertain  with  Gaussian  distribution.  The  objective  here  is 
to  determine  the  uncertainty  in  the  stability  of  system,  which  can  be  quantified  by  looking  at  the 
distribution  of  Xmax,  the  maximum  eigenvalue  of  the  Jacobian, 


J(a,  6, 


2axio 

2cxio 


2cx2o  ^ 
26x20 


where,  0:10,^20  the  equilibrium  satisfying 


+  cxlo  -v^  =  0, 

cxfg  +  bxlg  —V2  =  0. 


(69) 


(70) 

(71) 


Figures  below  show  result  of  probabilistic  waveform  relaxation  for  different  values  of  coupling  pa¬ 
rameter  c.  For  all  cases,  the  ground  truth  is  computed  based  on  collocation  on  the  parameter  space 
(a,  6)  with  I  =  10,  while  /j,  =  5.  Im  =  3  and  P  =  5.  In  all  cases  considered,  the  iterative  approach 
converges  to  the  ground  truth,  as  shown  by  the  histogram  of  Xmax  (see  figures  ??),  and  its  mean 
and  variance  (see  figures  ??).  As  the  coupling  strength  increases,  the  number  of  iterations  required 
for  the  convergence  increases. 


9  Conclusion  and  Future  Work 

In  this  paper  we  have  proposed  uncertainty  quantification  approaches  which  exploit  the  underly¬ 
ing  dynamics  and  structure  of  the  system.  In  specific  we  considered  a  class  of  networked  system, 
whose  subsystems  are  weakly  coupled  to  each  other.  We  showed  how  these  weak  interactions  can  be 
exploited  to  overcome  the  dimensionality  curse  associated  with  traditional  UQ  methods,  and  radi¬ 
cally  accelerate  uncertainty  propagation  in  large  systems.  By  integrating  graph  decomposition  and 
waveform  relaxation  with  generalized  polynomial  chaos  and  probabilistic  collocation  framework,  we 
proposed  two  scalable  iterative  UQ  approaches:  equation  based  which  requires  access  to  the  equations 
describing  the  underlying  dynamics  of  the  system,  while  the  other  one.  which  we  called  probabilistic 
waveform  relaxation^  treats  the  system  more  or  less  as  a  black  box.  The  second  approach  is  more 
practical  as  for  most  complex  networked  systems,  it  may  be  non  trivial  to  obtain  system  equations, 
if  not  impossible.  We  illustrated  the  probabilistic  waveform  relaxation  approach  on  a  simple  system 
with  promising  results. 

Many  questions  further  need  to  be  investigated.  First  of  all.  analytical  conditions  under  which 
the  two  iterative  schemes  proposed  in  this  paper  converge,  need  to  be  established.  The  choice  of 
collocation  parameter  (see  section  7.2)  plays  a  critical  role  in  how  much  computational  gain  can 
be  obtained  in  probabilistic  waveform  relaxation;  a  systematic  procedure  for  selecting  this  parameter 
is  therefore  crucial.  Finally,  this  algorithm  need  to  be  tested  on  a  larger  system,  to  establish  its  true 
potential  of  being  scalable. 
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A  Sparse  Grid  Methods 

A.l  Clenshaw  Curtis  Quadrature 

ClenshawCurtis  (CC)  quadrature  employs  a  change  of  variables  ^  =  cos^  and  uses  a  discrete  cosine 
transform  (DCT)  approximation  for  the  cosine  series,  in  order  to  compute  the  integral  in  Eq.  33. 
More  precisely,  the  cosine  series  expansion 


30 

g{cosO)  =  ^  +  Y^akCos(k6), 
k=\ 


(72) 


leads  to 

/  =  f  9(cos(0))w(cos(0))ii\n(0)d0 


=  "o  H-  ^  Cik^Ef: 

k=l 
P 


(73) 

k^\ 

where,  for  A:  =  0,  •  •  • , 

2  r 

ak  =  —  1  g [cos  0)co6(k6) do, 

^  Jo 

(74) 

and 

Wk  =  /  u;(cos6^)cos(Ar^)sin0c/^. 

Jo 

(75) 

Unlike  computation  of  arbitrary  integrals,  Fourier-series  integrations  for  periodic  functions  (like 
f(cosO)))  in  Eq  (74),  up  to  the  Nyquist  frequency  k  —  are  accurately  computed  by  the  p  equally- 
weighted  points 

=  cos(  ^  ^ k  =  1,“  -  (76) 

except  the  endpoints,  which  are  weighted  by  1/2,  to  avoid  double-counting.  With  this  the  integral 
(74)  can  be  approximated  as 


ak 


(77) 


For  most  w(^)j  the  integral  (75)  cannot  be  computed  analytically.  Since  the  same  weight  function  is 
generally  used  for  many  integrands  p(^),  however,  one  can  afford  to  compute  these  VVk  numerically  to 
high  accuracy  beforehand,  like  in  Gauss  quadrature.  Note  the  following  features  of  CC  quadrature: 

1.  Since,  by  definition  the  Chebyshev  polynomials  7\(^),  satisfy  Tk(cm6)  =  cos{kO)  CC  quadra¬ 
ture  can  be  thought  of  as  employing  the  expansion  of  the  integrand  (see  E}q.  72)  in  terms  of 
Chebyshev  polynomials  to  compute  the  integral. 

2.  In  CC  quadrature,  the  integrand  is  always  evaluated  at  the  same  set  of  points,  given  by  Cp 
(Eq.  76)  regardless  of  probability  density  function.  On  other  hand,  in  Gaussian  quadrature, 
different  density  functions  lead  to  different  orthogonal  polynomials  and  thus  different  roots 
where  the  integrand  is  evaluated. 

3.  For  g  =  2\z  >  1,  (7/  C  (this  would  be  the  notation  we  would  use  for  CC  ID  grid  from 
now  on,  i.e.  (7/  =  C'^(t))»  the  CC  quadrature  points  become  nested.  Gaussian  quadrature 
points  lack  this  property. 
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4.  The  CC  formula  is  less  accurate,  the  (g  -f  1)  point  CC  rule  can  provide  an  accurate  result  for 
integrating  polynomial  functions  of  order  up  to  g,  compared  to  -f  1  for  Gauss,  For  practical 
purposes,  however  both  method  lead  to  comparable  accuracy  [3],  This  is  possible  because 
most  numeric  integrands  are  not  polynomials,  and  approximation  of  many  functions  in  terms 
of  Chebyshev  polynomials  converges  rapidly. 

In  summary,  besides  having  fast-converging  accuracy  comparable  to  Gaussian  quadrature  rules, 
CC  quadrature  naturally  leads  to  nested  quadrature  rules,  which  is  important  for  both  adaptive 
quadrature  and  multidimensional  quadrature  (cubature).  There  are  other  quadrature  rules  with 
nested  properties,  details  can  be  found  in  [5], 

A. 2  Smolyak  Quadrature 

Smolyak’s  algorithm  is  a  method  first  developed  to  handle  high  dimensional  quadrature  [4]  and  later 
extended  to  accomplish  high  dimensional  interpolation  [6],  Its  basic  idea  is  to  use  the  solution  to 
several  low-dimensional  problems  to  span  the  space  and  then  linearly  combine  these  to  yield  the 
solution  to  higher  dimensional  problem  [5].  Let 

=  (78) 

be  the  difference  quadrature  formula  with,  =0.  In  general,  the  difference  are  therefore  quadrature 
formulas  on  the  union  of  grids  Cl  which  is  just  Cl  in  nested  case.  Based  on  these  difference 

formulas,  Smolyak  construction  approximates  the  integral  in  Eq.  (4)  by 

S(l,p)[9]=  (A{,  ®  Aj,  •••  ®  A‘^)lg],  (79) 

|i|<i+p-i 


where,  |i|  =  ii  -f  h  *  This  can  be  expressed  in  terms  of  U\^ ,  as 

S(l,p)[9]=  ^  (80) 


where, 

(p-i\  ^  (P-  1)! 

VN-V  (|i|-0!(p-|i|  +  0!’ 


(81) 


is  the  factorial.  Prom  above  equation  we  see  that  like  full  tensor-quadrature,  Smolyak  quadrature  for¬ 
mulas  are  special  tensor-product  rule  w'hich  are  constructed  from  tensor  products  of  one-dimensional 
quadrature  formulas,  but  these  are  combined  so  that  in  only  some  dimensions  quadrature  formulas 
of  high  order  are  used  while  formulas  of  lower  order  are  used  in  the  other  dimensions.  One  could 
also  write  above  formula  in  recursive  fashion,  like 


S{l,p){9]  =  ^S(l-k,p-  l))[p],  (82) 

and 

S(l  +  \,p+\)[g]=  Y.  (A.'.®A.V--®A>^®W,Vp_|,|)[g].  (83) 

|i|</+p-l 

Note  that  to  compute  (79),  w^e  only  need  to  evaluate  function  g  over  so  called  sparse  grid,  which  is 
given  by  union  over  the  pariwise  disjoint  grids  Si^  x  •  •  •  x  5jp. 

S(l,p)=  U  x---x5i^,  (84) 

|i|</+,.-i 
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where,  x  denotes  the  usual  Cartesian  product.  For  non  nested  cas*e 

S,=Cl,  (85) 

while,  for  nested  case 

S,  =  Cl\Cl_^,  (86) 

with  Cq  =  0,  where  recall  Cl  is  a  set  of  ID  quadrature  points  corresponding  to  level  k.  Thus, 
unlike  full  grid  (Eq.  40),  the  sparse  grid  is  a  union  of  several  tensor  products.  Based  on  Eq.  (80), 
an  appropriate  choice  of  vector  i  =  »*p)  gives  level  of  accuracy  in  each  dimension,  from 

which  the  sparse  grids  are  obtained  by  usual  product  i.e.  *5*1  x  Si,  •  •  In  case  the  univariate 
formulas  are  nested,  the  sparse  grids  are  also  nested,  i.e. 

S(l,p)cS(Uhp).  (87) 

The  number  of  points  Q  in  sparse  grids  S(l^p)  are  given  by 

Q=  E  Pm  •••Pip-  (88) 

where, 

Pfc=Card(5fc),  (89) 

is  the  cardinality  i.e.  number  of  points  in  the  set  Sk-  If  m/  =  0(2^)  the  order  of  Q  is 

Q  =  0(2^C-^)  (90) 

This  shows  that  the  dependence  on  dimension  is  much  weaker  on  n  the  number  of  uncertain  param¬ 
eters,  compared  to  0(2^^)  for  full  grid.  Table  1  gives  a  comparison  of  the  number  of  grid  points  in 
different  schemes. 

1  For  n  >  5  the  sparse  grid  methods  prove  significantly  more  advantageous  than  full  grid. 

2  Nested  quadrature  rules  lead  to  sparser  grids  compared  to  non-nested  quadrature  rule. 

3  The  asymptotic  accuracy  of  sparse  grid  method  is  comparable  to  that  of  full  grid. 

More  precise  results  on  accuracy  and  convergence  properties  of  Smolyak  grids  with  different  quadra¬ 
ture  rules  can  be  found  in  [5];  for  multielement  formulation  of  sparse  grid  see  (2). 

In  summary,  the  Smolyak  formula  (Eq.  80)  can  be  expressed  as 

V^,  P.p 

-‘’(^>p)[9]=  E  E  ■■■  E 

iii<i+p-iji=i  j„=i 

with,  i  =  (iuh,---  -i/O.j  =  (juj2,---  ,jp),  and  ry  =  (n.^,,--  -  ,ri„j„).  Again  note  that  i  = 
•  •  ^ip)  is  vector  specifying  levels  of  quadrature  formula  in  each  dimension.  For  each  i/t,  Pi^ 
(see  Eq.  89)  denotes  the  actual  number  of  points  in  the  ID  grid  (Sk  see  Exj.  85  or  86)  in  k-th 
dimension;  hence  jk  goes  till  to  cover  all  points  in  the  grid  in  each  dimension. 

For  non  nested  case,  weights  are  combined  as  follows 

VVy  =  (92) 

while  in  nested  case, 

^^  0“  ’  ^(»n+flp)jn»  (93) 

|i+q|<Z+2ri-l 
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Random  Dimension  (n) 

Level  (/) 

FG 

SG  CC 

SG  Gauss 

3 

2 

8 

7 

10 

3 

27 

25 

52 

4 

64 

69 

195 

5 

125 

177 

609 

6 

215 

441 

1710  ■ 

7 

343 

1,073 

4502 

5 

2 

32 

11 

16 

3 

243 

61 

131 

4 

1.024 

241 

746 

5 

3,125 

801 

3376 

6 

7,776 

2,433 

13,083 

7 

16,807 

6,993 

45,458 

8 

32,768 

19,313 

145,873 

9 

59,049 

51,713 

440.953 

10 

100,000 

135,073 

1,272,848 

10 

2 

1,024 

21 

31 

3 

59,049 

221 

486 

4 

1,048,576 

1,581 

5166 

5 

9,765,625 

8,801 

42,101 

6 

60,466,176 

41,265 

281,867 

20 

2 

1,048,576 

41 

61 

3 

3,486,784,401 

841 

1871 

4 

1,099,511,627,776 

11,561 

38,531 

5 

95,367,431-640,625 

120,401 

600,226 

Table  1;  Comparison  of  full  gird  and  sparse  grid  PCM  using  different  quadrature  rules.  In  order 
to  generate  above  table,  =  /  -I-  1  for  Gauss,  while  m}  =  2^”*  +  !,/>!  (and  m}  =  1,/  =  1)  for 
nested  CC  univariate  quadrature  formula  (see  Elq.  36).  This  table  has  been  partially  taken  from  [9], 
with  last  column  generated  using  MEl-gPC  code  (see  next  section). 
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with  q  €  N"  and 


(94) 


where,  s,r  are  such  that  Vkj  =  r^k+q-\)r  -  rik+q-2U- 

The  weights  can  be  precomputed  in  both  cases,  so  that  there  is  no  practical  difference  concern¬ 
ing  the  overall  cost  of  the  quadrature  formula.  Note  that  Smolyak  formulas  can  contain  negative 
weights  even  if  the  underlying  univariate  quadrature  formula  have  positive  weights.  Convergence 
is  guaranteed,  because  values  of  the  weights  remain  relatively  small.  For  more  details  on  stable 
numerical  implementation  of  Smolyak  quadrature,  the  reader  is  referred  to  [5]. 
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Figure  1:  Convergence  of  distribution,  c  =  0.1 


Figure  2:  Convergence  of  mean  and  variance 
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Figure  3:  Convergence  of  distribution,  c  =  1.0 


Figure  4:  Convergence  of  mean  and  variance 
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Figure  5:  Convergence  of  distribution,  c  =  2.0 


Figure  6:  Convergence  of  mean  and  variance 
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Figure  7:  Convergence  of  distribution,  c  =  2.8 


Figure  8:  Convergence  of  mean  and  variance 
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B.2  Uncertainty  propagation  by  various  methods 
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Summary 


The  main  results  presented  in  this  work  are  summarized  next. 

PDF  Fitting  for  Uncertainty  Propagation,  Long  Tail  input  PDFs,  High 
dimensions.  Unknown  Unknowns.  The  approximation  of  PDFs  of  outputs  of 
systems  that  depend  on  stochastic  inputs  can  be  used  to  estimate  statistical  measures 
of  the  outputs.  These  PDFs  can  be  used  to  propagate  further  the  uncertainty  in 
complex  systems.  Computations  of  intermediate  PDFs  and  merging  of  PDFs  can 
provide  large  computational  savings  in  uncertainty  propagation  (UP)  computations 
for  complex  systems.  Formulas  that  evaluate  these  computational  savings  are 
provided.  Methods  of  fitting  PDFs  are  illustrated  on  several  UP  problems  as:  a) 
Computation  of  phase  change  temperatures  depending  on  4  input  parameters  with 
long  tail  PDFs;  b)  In  high  dimension  examples,  PDFs  with  up  to  2000  input 
parameters  are  effectively  estimated;  c)  The  effectiveness  of  estimating  PDFs  is 
illustrated  for  high  dimension  cases  that  also  include  unknown  unknowns(inputs)  with 
unknown  PDFs. 

Optimized  Interpolation  for  parameters  and  number  of  eigenvectors  using 
probability  weighted  objectives  is  an  interpolation  procedure  that  combines  a)  a 
Nystrom  kernel  based  interpolation  approach  with  b)  an  optimization  of  the 
parameters  of  the  kernel  such  that  to  minimize  the  difference  between  statistical 
measures  of  the  interpolated  function  and  of  the  data.  This  method  was  used  in  the 
response  surface  (RS)  UP  approach. 

Domain  Exploration  approaches,  uniform  domain  covers,  hierarchical  stochastic 
covers  for  Optimization  and  UP.  Finding  iX)sitions  of  points  that  provide  quasi 
uniform  domain  covers  are  important  for  designs  of  experiments,  optimization,  UP, 
interpolation,  domain  discretization,  solving  PDFs  etc.  Domain  exploration 
approaches  are  more  and  more  often  used  in  industrial  applications  where  an 
understanding  of  the  space  of  possible  designs  is  thought  for,  e.g.,  for  regions  of 
robust  solutions,  or  for  optimal  solutions  with  respect  to  multiple  objectives.  Domain 
exploration  approaches  usually  offer  more  information  about  the  design  space  than 
optimization  approaches.  Finding  good  sets  of  sampling  points  is  important  for  the 
efficiency  of  domain  exploration  approaches  and  for  UP.  It  is  desired  to  obtain  the 
maximum  amount  of  information  about  the  design  space  using  a  smallest  number  of 
samples.  Simple  and  robust  hierarchical  stochastic  cover  techniques  are  presented.  A 
repelling  particle  method  is  illustrated  where  n  points  in  a  domain  that  act  as 
repelling  particles  pushed  by  potentials  that  aim  to  enforce  given  properties  such  as 
uniform  spacing.  The  particles  may  be  denser  in  regions  where  given  PDFs  are  higher 
(the  PDFs  are  treated  as  separate  potentials).  Treatment  of  boundaries  and  constraints 
in  searches  and  optimization  is  performed  by  three  techniques:  direct  enforcing  of  the 


109 


B.2.  UNCERTAINTY  PROPAGATION  BY  VARIOUS  METHODS 


constraint,  using  of  boundary  particles  and  using  of  boundary  potentials  that  repel  the 
particles  towards  the  interior.  We  discuss  a  Hierarchical  Architectural  Optimization 
approach  using  these  ideas.  Results  for  Domain  Exploration,  Global  Optimization  and 
finding  Multiple  Local  Minima  by  Adaptive  Hierarchical  Repelling  Particle 
Techniques  are  demonstrated.  In  addition,  it  is  demonstrated  that  the  optimization 
will  track  the  found  local  minima  in  a  case  when  the  objective  varies  continuously  in 
time  (or  depending  on  parameters),  i.e.,  the  problem  has  a  dynamic  objective. 


A  Generic  Unstructured  Multigrid  (MG)  Approach  in  High  Dimensions  is 
proposed  that  combines  the  presented  domain  covering  approaches,  with  the 
interpolation  approaches,  and  with  known  multigrid  formulations.  These  generalize 
known  MG  techniques  to  sparse  data  in  high  dimensions  and  combine  the  efficiency 
of  hierarchical  structures  with  local  iterations  and  with  approximation  of  solutions  by 
reduced  models. 

A  Hierarchical  Richardson  Approach  for  Model  Extrapolation  is  suggested. 
Coefficients  of  a  sequence  of  models  are  identified  by  system  identification  and 
extrapolated  as  in  a  Richardson  procedure.  Models  may  be  decomposed  into 
deterministic  sub-models  (e.g.,  trends)  and  stochastic  sub-models  (e.g.,  noise).  Model 
extrapolation  can  be  used  for  both  deterministic  and  stochastic  mcxiels,  hence  for  UP. 
Large  computational  savings  may  be  obtained  by  extrapolating  a  sequence  of  coarse 
model  results  to  approximate  fine  model  results. 

A  Response  Surface  (RS)  Uncertainty  Quantification  Approach  is  compared  with 
MC,  a  Polynomial  Chaos  Collocation  approach,  and  with  DSAMPLE  (an  effective 
quasi  MC  technique)  on  the  Kr  Milestone  Problem.  Advantages  and  disadvantages 
of  the  RS  approach  are  discussed. 

A  Sum  of  Gaussians  State  Uncertainty  Propagation  method  was  demonstrated  for 
a  New  England  Power  Grid  model  and  the  computational  time  savings  using  the 
proposed  merging  of  PDFs  approach  is  discussed. 


1.  PDF  Fitting  for  Uncertainty  Propagation,  Long  Tail  input  PDFs, 
High  dimensions,  Unknown  Unknowns 

The  approximation  of  PDFs  of  outputs  of  systems  that  depend  on  stochastic  inputs  can  be 
used  to  estimate  statistical  measures  of  the  outputs.  These  PDFs  can  be  used  to  propagate 
further  the  uncertainty  in  complex  systems.  It  is  shown  that  appropriate  computations  of 
intermediate  PDFs  can  significantly  reduce  the  complexity  of  uncertainty  propagation  in 
(UP)  computations.  Methods  of  fitting  PDFs  are  illustrated  on  several  UQ  problems  as; 
a)  Computation  of  phase  change  temperatures  depending  on  4  input  parameters  with  long 
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tail  PDFs;  b)  In  high  dimension  examples,  PDFs  with  up  to  2000  input  parameters  are 
effectively  estimated;  c)  The  effectiveness  of  estimating  PDFs  is  illustrated  for  high 
dimension  cases  that  also  include  unknown  unknowns(inputs)  with  unknown  PDFs. 

The  main  idea  is  to  approximate  the  PDF(0  by  fitting  a  histogram  of  f(x)  as  shown  in 
the  figure  below. 


Multiple  techniques  have  been  applied  such  as  sums  of  Gaussians  and  kernel  density 
functions.  For  example,  the  PDF(f)  may  be  approximated  a  by  a  sum  of  Gaussians  or  of 
other  basis  functions  (such  as  rectangles),  or  a  spline  fitting  a  histogram  etc. 

If  the  PDF(f)  is  smooth,  then  a  small  number  of  points  may  be  sufficient  to  approximate 
it  well.  The  approximation  of  the  PDF(0  is  an  ID  problem,  although  f(x)  may  depend  on 
the  variables  x  in  high  dimensions.  Moreover,  the  fitting  does  not  use  the  PDF(x). 

Even  more,  the  PDF(f)  may  depend  on  many  unknown  unknowns  with  unknown  PDFs. 
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PDF(T)  Approximations  by  Kernel  Density,  Long  Tail  4D 

The  parameters  and  their  PDFs  are  not  used 

Works  in  high  dimensions  for  any  number  of  parameters  and  with 

unknown  distributions;  Works  for  unknown  unknowns 

Model:  T(x)  =  Gaussi  *  Regl  +  Gauss2  *  Reg2  fitting  MD  data,  x  In  4D  LT 


V 


50K  samples: 
mean_T_reg  MC  =  1 06.65 
var_T_reg  MC  =  400.02 
mean_T_bln  Hist  =  106.65 
var_T_bln  Hist  =  400.02 
m©an_TE_ker  GSS  =  106.65 
var  TE  ker  GSS  =  400.31 


Convergence  of  Mean  and  Var 


PDF  of  T  Bln<r),  k«r_0S8<c^  k«’_BOX{b) 


PDF(T) 

Approximations 
50K  samples 


::pdf(t)  / 

Approximations,' 
500  samples  ? 


j  > 
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PDF(T)  for  500  Samples 


i  Probability  of  p(x)  for 

...  !\  different  x  plott^  for  T(x) 

;  !!!  T(x)=T(y)  may  accumulate 

;  1  2  different  probabilities  : 

M  p(,x)i^p{y)  for  x^y 

^  /v-.  ^ ..  ^  _  V;. 

PDF(T)  ~  Sum_x  g(T(x),  T)  /  N 
Sum  of  Boxes,  Gaussians,  Kernels 
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DARPA  Kr  Milestone  4  Gaussian  PDFs  (5000  samples) 

The  PDF  statistical  measures  reflect  the  Measures  of  the  Samples 

The  quality  of  the  sampling  is  reflected  in  the  quality  of  the  PDF 

and  they  should  have  similar  convergence  rates, 

e.g.,  DS  +  PDF  should  converge  faster  than  MC  +  PDF 

!!  The  time  savings  come  when  Uncertainty  is  Propagated  using  PDFs 


PDF  Of  T  Bln(r).  k#r_OSS(c).  k*r_BOX(b) 


mean  MC  =  110.13 
var  MC  =  309.04 

mean  PDF  =  110.13 
var  PDF  =  317.45 

Analytic  results  Vladimir: 
mean!  An:  110.30 
var  T  An:  315.61 

The  Kernel  PDF  (blue)  is  smoother 
than  the  Histogram  (red) 
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High  Dimensions,  Unknown  Unknowns 
2000  parameters  (Normal  PDFs) 

“unknown”  parameters  random  (uniform)  drive  the  mode  switch 
500  sampies,  10''(-3)  relative  error  MC  vs  PDF 
5000  samples,  10''(-4)  relative  error  MC  vs  PDF 


■  POFefT  Mn(r).lnr  .OSS<c).tMr  .BOX(b) 


500  samples 

mean  MC  = 

186.53 

mean  GSS  PDF  =  186.54 

var  MC  = 

1.9002e+005 

var  GSS  PDF  = 

:  1.9021e+005 

5000  samples 

mean_MC  = 

183.35711 

mean_PDF  = 

183.35707 

var_MC  = 

185579.5 

var_PDF  = 

185542.8 

Propagation  of  PDFs  in  complex  Systems 
Complexity  reduction  by  merging  PDFs 

The  propagation  of  PDFs  in  complex  systems  by  computations  of  intermediate  PDFs 
and  merging  of  PDFs  can  significantly  reduce  the  complexity  of  uncertainty 
propagation  in  (UP)  computations.  The  following  schemes  provide  the  computational 
time  saving  using  PDF  merging. 
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High  Compl9xlty  of  approximating  PDFs  by  MC/OuasI  -  WC, ... 


PDF(z) 

A 


T1  BTIm«tOMllmM«POf<z)dlr«otty  Iromxl,  k2  by  MC/OMC,... 

T1  s  (  N^vartables  *  N_samp(M)  ‘  (Tf  ♦  Tg)  ;  Tt.Tg  -  BvaiuM*  t,  g 
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Time  Saving  by  Merging  PDFs 


Besides  computing  y  also  compute  PDF(y): 
Merge  N^variable  PDFs  Into  1  PDF 


PDF(z) 


A 


T1  ■  (  N_vari«bles  *  N  samples)  *  (Tf  -f  Tg)  ;  Tf,  Tg  >  time  to  evaluate  f,  g 
T2  ■  Time  to  estimete  PDF(z)  from  x1,  x2  computing  Intermediate  PDF(y): 

T2  s  (  N_varlebles  *  N_samples)  *  Tf  ♦  N  samples  *  Tg, 


Time  Savings  in  epproximating  PDP(2)  >  T1  -T2  «  (  N  variables  -1)  *  N  samples  *  T^^ 
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Time  Saving  by  Merging  PDFs  +  Reducing  MC  to  PDF  Fitting 


Besides  computing  y  also  compute  PDF(y): 

Merge  N_varlabie  PDFs  into  1  PDF  -f  Replace  MC  for  y  by  PDF  Fitting 


PDF(z) 

A 


T1  m  TIrm  to  aotirml*  PDf(z)  d(r»ctly  from  x1,  x2  by  MC !  QMC, 

T1  «  (  N_vart«btoa  •  N^MinpIta)  ■  {Tf  Tg)  ;  Tf.  Tg  «  tlm#  to  •vakiaia  f,  g 
T2  m  Ttma  to  ••tfmat*  PDF<z)  from  x1,  x2  computing  Intermadiate  PDF(y): 

T2  ■  ( N_var1ablaa '  N_aampfaa)  *  Tf  ♦  N_PDFfit  '  Tg. 

Time  Savings  m  approximating  PDF(z)  .  T1  -T2  ■  Tg  *  (  N  variables  *  N. samples  •  N  PDFflt) 
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Time  Saving  by  Merging  PDFs  +  Reducing  MC  to  PDF  Fitting 


Besides  computing  y  also  compute  PDF(y): 

Merge  N_variabie  PDFs  into  1  PDF  Replace  MC  for  y  by  PDF  Fitting 


PDF(x1) 

A 


N_var1ablaa 

N_aamplaa 

PDF(x2) 

A 


z=g(f(x1,x2)) 


PDF(z) 

A 


T1  >  Tima  to  aatlmata  PORz)  dlractfy  from  x1,  x2  by  MC  /  OMC, ... 

T1  m  (  N_vartablaa  *  N_aamplaa)  *  (Tt  *  Tg)  ;  Tt,  Tg  ■  ttma  to  avaluala  f,  g 
T2  ■  Tima  to  aatlmata  PDF(z)  trom  x1,  x2  computing  Intarmadlata  PDF(y); 

T2  .  (  N  variabla  PDFfh)  •  Tf  +  N^PDFflt  *  Tg, 

Time  Savings  m  approximating  pdF(z)  ■  ti  -  t2  ■ 

Tt*  (  N  variables  *  N  samples  -  N  variable  PDFflt)  Tg  *  ( N  variables  *  N  samples  •  N  PDF^ 


2.  Optimal  Interpolation:  Interpolation  optimized  for  parameters  and 
number  of  eigenvectors  using  probability-weighted  objectives. 
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The  presented  interpolation  procedure  that  combines  a)  a  Nystrom  kernel  based 
interpolation  approach  with  b)  an  optimization  of  the  parameters  of  the  kernel  such  that  to 
minimize  the  difference  between  statistical  measures  of  the  interpolated  function  and  of 
the  data.  This  method  was  used  in  the  response  surface  (RS)  UP  approach. 


Response  Surfaces:  Nystrom  Interpolation  with  PDF  weighted  Optimization 

Fit  functions  using  Eigenvectors  of  a  Kernel  as  basis  functions 


Given  ihc  values  fix,)  of  /  at  the  points  fx^ . jr,/. 

approximat  ^fjc)  and  derivatives  of  /  at  x. 


1)  Kv  =  A,v,  K  is  a  kernel,  e.g.,  v)  =  exp(-ll  jf- yll' /a*) 


3)  v(^)  =— extend  the  eigenvectors  using  K 


4)  /(Jf,)  fit  f  by  a  PDF  weigh  ted  Opti  mi  zat  ion  (see  next)  using  a  subset  of  eigenvectors 


5)  fix)»»^a^Vj{x)  extend  /  using  extension  of  eigenvectors  3) 


Optimization  Added  to  Nystrom 


6)  = 


7)  =  get  derivatives  of  f  from  derivatives  of  K 


V. 
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Optimization  of  the  RS  to  fit  the  data,  the  mean  and  var  of  the  data 
The  samples  x  are  fixed,  Rnd  the  RS,  R(x)  that  best  fits  the  data  f(x)  by  varying  the 
number  of  eigenvectors  (M),  kernel  parameters  (sigma),  and  coefficients  (a)  of  the 
eigenvector  expansion 


Nystrom:  Given  ihe  N  values  fix,)  of  /  ai  the  N  points  /jc, . x,,/. 

approximate  fix)  at  x. 

1)  Kv  =  A.v.  K  is  a  kernel,  e.g..  Ar(jr.y)  =  exp(-ll /tr^)  Ny.N 

M 

2)  /(.V, )« )  =  )  approximate  f  using  a  subset  of  M  eigenvectors  and  an  Optimization  as: 

3)  O,  =  mint  min  II  / -/?!!/ II /II )  forAf,  a,  a.  gives  good  mean  and  var  approximations 


4)  Oj  =min(minl^l/(jc,)-/f(jr, )l  p(.r,)/(^p(jc,)+  wl  var(f)- var(R)l  |} 
minimizez  mean  and  variance  errors  using  known  PDFs 

5) 0,  =  min{  mini  mean,  var,  PZ)F(/fS)ll 

M  <T  • 

insures  R  converges  to  f  in  normt  and  ...PDF),  where  PDF{RS)  is  an  approximation  of  the  histogramtf) 

6)  Bootstarp  approaches:  compare  N  different  RS's.  each  for  another  subset  of  N-1  samples 


Several  potential  advantages  of  RS  for  uncertainty  quantification 


1)  The  RS  approach  may  work  in  large  dimensions  (10'^2  -  lOM  ?)  for  RS  a  small 
number  of  "good"  eigenvectors  may  be  sufficient.  The  eigenvectors  may  represent 
well  the  reduced  dimensionality  of  the  data. 

2)  The  RS  approach  may  be  easily  applied  on  general  domains  and  on  manifolds 

3)  The  RS  approach  may  use  a  relatively  small  number  of  evaluation  points  (e.g.,  quasi 
uniform  in  high  dimensions) 

4)  The  eigenvectors  used  in  RS  may  depend  on  the  problem  -  an  advantage  in 
approximating  features  of  the  surface 

5)  The  eigenvectors  may  have  a  local  character  (e.g,,  sums  of  Gaussians)  hence  may 
represent  local  features  of  surfaces 
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6)  Error  and  convergence  estimates  based  on  eigenvectors  and  eigenvalues  may  be 
provided 

7)  The  RS  approach  is  general:  any  probability  distributions  of  parameters  may  be  used 
for  the  RS 

8)  The  RS  approach  may  also  be  used  when  unknown  unknowns  (with  unknown 
distributions)  are  present  (as  shown  in  our  approach,  where  about  lOM  random  initial 
positions  and  initial  velocities  are  present),  (see  the  extrapolation  approach  that  estimates 
models  of  noise). 

9)  Extension/Extrapolation  using  eigenfunctions  may  have  improved  reliability  for 
bounded  functions. 

10)  RS  provide  multiple  altematives/freedom  in:  selecting  the  kernels,  evaluation  points, 
handling  convergence,  handling  high  dimensions,  handling  general  domains  etc. 


Disadvantages  of  Eigenvector  based  RS  Approaches 

An  expensive  diagonalization  of  a  large  matrix  is  required.  The  matrix  may  be  sparse  but 
it  is  not  banded  usually.  Eigenvalue  computing  time  and  accuracy  are  major  concern 
issues.  Numerical  instabilities  may  happen  due  to  small  eigenvalues,  especially  in 
schemes  that  involve  division  by  the  small  eigenvalues  (that  often  have  large  relative 
errors).  This  difficulty  may  be  handled  by  the  hierarchical  RS  approaches.  The  selection 
of  the  kernels  and  of  parameters  of  kernels  are  not  obvious.  It  is  not  easy  to  define  local 
parameters,  although  the  structure  of  the  problem  may  require  this,  e.g.,  a  Gaussian  sigma 
should  depend  on  the  density  of  the  data. 


3.  Domain  Exploration,  Hierarchical  Uniform  Domain  Covers  for  UQ. 
Hierarchical  Architectural  Optimization. 


Finding  positions  of  points  that  provide  quasi  uniform  domain  covers  are  important  for 
designs  of  experiments,  optimization,  UP,  interpolation,  domain  discretization,  solving 
PDEs  etc.  Domain  exploration  approaches  are  more  and  more  often  used  in  industrial 
applications  where  an  understanding  of  the  space  of  possible  designs  is  thought  for,  e.g., 
for  regions  of  robust  solutions,  or  for  optimal  solutions  with  respect  to  multiple 
objectives.  Domain  exploration  approaches  usually  offer  more  information  about  the 


120 


B.2.  UNCERTAINTY  PROPAGATION  BY  VARIOUS  METHODS 


design  space  than  optimization  approaches.  Finding  good  sets  of  sampling  points  is 
important  for  the  efficiency  of  domain  exploration  approaches  and  for  UP.  It  is  desired  to 
obtain  the  maximum  amount  of  information  about  the  design  space  using  a  smallest 
number  of  samples.  Simple  and  robust  hierarchical  stochastic  cover  techniques  are 
presented.  A  repelling  particle  method  is  illustrated  where  n  points  in  a  domain  that  act 
as  repelling  particles  pushed  by  potentials  that  aim  to  enforce  given  properties  such  as 
uniform  spacing.  Treatment  of  boundaries  and  constraints  in  searches  and  optimization  is 
performed  by  three  techniques:  direct  enforcing  of  the  constraint,  using  of  boundary 
particles  and  using  of  boundary  potentials  that  repel  the  particles  towards  the  interior. 

The  approach  is  used  for  Hierarchical  Architectural  Optimization. 


In  the  hierarchical  stochastic  covering  approach,  a  domain  is  covered  stochastically  by 
non-intersecting  balls  of  same  radius.  A  sequence  of  covers  of  decreasing  ball  radii  is 
used  as  shown  below.  Stochastic  stopping  criteria  are  proposed. 

Similar,  using  a  particle  repelling  approach  to  cover  a  domain  by  repelling  particles,  for 
example  repelled  by  Gaussian  potentials  of  given  width,  a  hierarchical  cover  is  obtained 
by  selecting  Gaussians  of  different  widths. 


Applications: 

1 .  UQ  for  selecting  sampling  points 

2.  RS,  Interpolation,  Fitting 

3.  Design  of  Experiments  (DOE),  Domain  Exploration 

4.  Global  Optimization  using  hierarchical  approaches 

Advantages: 
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1 .  Works  in  high  dimensions 

2.  Uniform  full  domain  cover. 

3.  Avoids  meshing  -  griding 

4.  Works  on  unstructured  domains,  manifolds 

5.  Multiscale  covering 

6.  Handling  of  Boundaries,  Constraints 
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Domain  Exploration-DOE  using  Repelling  Potentials  and  Particle  Dynamics 

Particles  driven  by  a  repelliriQ  potential  cover  uniformly  an  unstructured  domain 

Ronald  Colfman  (Yale),  Mihal  Putinar  (UCSB) 


X(t+1)  =  X(t)  -  dt  *  grad(P(X(t)) 
direct:  min(P(X(t)) 


Design  of  Experiments  (DOE)  Problem: 

Position  K  points  uniformly  inside  a  general  domain. 

Approach:  set  K  initial  particles  randomly  inside  the 
domain  and  drive  them  by  a  dynamics  based  on  a 
repelling  potential.  The  steady  state  provides  a 
"uniform"  covering  solution.  Handle  boundaries  and 
constraints  by  potentials. 


Advantages  compared  to  other  DOE  techniques: 

1 .  Any  number  K  of  points  may  be  distributed,  even 
for  very  small  K  >s  2,  In  large  dimensions. 

2.  The  distribution  of  points  Is  uniform 

3.  The  domain  may  be  unstructured  (e.g.,  not  a  box) 
or  a  manifold 

4.  Constraints  may  be  handled  by  potentials. 

5.  More  efficient  than  Latin  Hypercube  on  general 
domains  and  manifolds 

6.  Additional  structure  may  be  Imposed  using  Interior 
potentials  representing  probabilities  or  regions  of 
Interest. 


5 


Quasi  Uniform  Distribution  of  Points  by  Repelling  Potentials 
The  Boundary  Points  are  kept  fixed 

Final  Particle  Positions 

Iniatial  Particle  Positions 
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Exploration  Approaches;  Hierarchical  Architectural  Optimization 

Hierarchical  Architectural  Optimization  finds  best  Configurations  satisfying  a  set  of  Objectives  and 

Constraints 


Goal:  Build  a  Dictionary  of  Good  Architectures/ Designs/Conligu rations, 
e.g.,  for  different  costs,  performances,  weights,  design  options. 

How: 

Build  a  Hierarchy  of  Clusters:  Poor  Clusters  are  eliminated 
starling  with  Coarse  Large  Clusters 

Good  Clusters  are  refined  Hierarchically 

Elimination  Criteria  are  added  as  new  Constraints  during 
the  Optimization  by  Analysis  of  Solutions 

Continuous  Optimization  is  performed  only  close  to  the  set 
of  best  solutions 


Highlights  of  Results 

•Orders  of  magnitude  savings  in  expehmental 
work  and  lime 

•Reliable  Global  optima  for  exhaustive 
hierarchical  searches  (10000  times  reduction  in 
number  of  configurations) 

•Very  high  confidence  that  best  solutions  have 
been  found 

•Can  search  all  possible  discrete  combinations 
in  finite  unstructured  domains,  (with  no 
deterministic  combinatorial  algorithm) 


Main  techniques  used: 

1.  Hierarchical  Optimization 

2.  Elimination  Crtteiia  added  adaptively 

3.  Global  Searches  for  Multiple  Objectives 

4.  Local  Continuous  Optimization 


Domain  Exploration,  Global  Optimization  and  finding  Multiple  Local  Minima 
by  Adaptive  Hierarchical  Repelling  Particle  Techniques. 

The  figure  below  illustrates  a  domain  exploration  and  global  optimization  approach. 
The  objective  is  changing  in  time  (it  rotates  in  this  example)  and  presents  two  large 
local  minima  to  be  found  and  tracked  in  time.  A  hierarchy  of  optimizations  is 
performed  by  a  hierarchy  of  potentials.  The  potentials  generate  broad  large  step 
searches  at  start,  in  a  global  domain  exploration  phase,  and  then  small  step  local 
searches  close  to  local  minima.  Boundary  constraints  are  implemented  by  boundary 
potentials,  in  this  case,  the  boundary  potential  keeps  the  particles  inside  the  circular 
domain. 
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Domain  Exploration,  Hierarchical  Global  Optimization  and 
finding  Multiple  Local  Minima  for  Time  Dependent  Objectives 

Two  large  local  minima  vary  In  time. 

Tba  Hlenrchy  oi  Optimizitions  Is  performed  by  •  Hierarchy  of  Potentials  that  gerwrata  broad  largo 
atop  searchas  at  atari  (Domain  Exploration  phaaa)  and  then  email  atap  local  searches  close  to  local 
minima. 

Tha  objective  la  time  depertdent  (It  rotaiea  In  thia  example) 

Boundary  Conatraints  are  Implemented  by  Boundary  f\>tentlals. 


Potential  for  43  particies 
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4.  An  Unstructured  Multigrid  Approach  in  High  Dimension 

A  Generic  Unstructured  Multigrid  (MG)  Approach  in  High  Dimensions  is  proposed 
that  combines  the  presented  domain  covering  approaches,  with  the  interpolation 
approaches,  and  with  known  multigrid  formulations.  These  generalize  known  MG 
techniques  to  sparse  data  in  high  dimensions  and  combine  the  efficiency  of  hierarchical 
structures  with  local  iterations  and  with  approximation  of  solutions  by  reduced  models. 

Multilevel  approaches  try  to  accelerate  the  solution  of  a  problem  by  other  problems  on 
different  levels,  for  example  by  coarser  representations  of  the  initial  problem. 

The  multilevel  approaches  comprise  two  main  procedures: 

1)  solvers  of  each  problem  on  its  level; 

2)  inter-level  transfers  of  data,  variables,  and  operators. 
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Problem  formulation : 


L(x,u)  =  b,  jc:  points  in M  unknown  variables  at  jc 
b  known  data  at  x,  L:a  general  operator 

Inter-  level  Transfers 

LetX^  and  be  two  sets  of  points 

The  transfer  of  u' ,b'  fromX^  to  X"' is  performed  by  Interpolation 
b^=l'-'b' 


Coarse  Level  Operator 

Given  theoperator  Ll  define  the  operator  L!  by  an  approximation  of  U  ' 


Coarse  Level  Problem  (Full  Approximation  Scheme  (FAS)) 

Given  the  level  I  approximate  solution  h'  of  Ll  (X'  ,u')  =  b' 

Define  the  level  J  term  b^  and  level  J  problem  formulation  by 

Ll{X\u')  =  b'  =U  {X\l'-\')  +  l'\b' -lJ{X'  ,u')) 

Update  of  Fine  Level  Solution 

The  (FAS)  correction  of  level  1  solution  u'  by  level  J  solution  is: 

Single  Level  Solvers,  Single  Level  Relaxation 
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Iteratively  update  subsets  of  variables  e.g.,  by  local  optimization  or  Newton  approaches 
as  follows. 

Problem  formulation  on  a  given  level:  L(x,u)  =  b, 

X :  points  in  R"  indexed  by  /,  X  \ 

u  unknown  variables  at  jc,/?  known  data  at  jc,  L :  a  general  operator 

Local  update  of  solutions 

Let  bea  subset  of  the  points  X^ 

The  associated  equations 

(1)  i! {X\u^ aresolvedapproximately  forw^  by : 
an  optimization  or  solver  procedure  (e.g.,  Newton) 

(2)  The  equation  (1)  is  solved  for  different  subsets  of  indices  J  of  I 
until  a  convergence  criterion  is  satisfied  (Block  Newton  Gauss -Seidel) 

If  y  =  /  then  a  direct  solver  may  be  used,  e.g.,  on  coarse  levels. 

This  procedure  is  a  block  type  Gauss-Seidel  iterative  relaxation  procedure. 


A  Local  Interpolation  Approach  using  a  Partition  of  Unit 
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Let  X  be  a  finite  set  of  points  and  f:X  — >  R  with  known  values  on  X 
Let  be  subsets  of  X  indexed  by  the  indices  I 
Let  approximate  or  interpolate  f(x)  onX\  e.g.,  be  local 
Let  p'  be  functions  equal  to  0  outside  a  set  containing 


The  interpolation/fitting  has  a  local  form : 
f(x)=g(x)  =  2^/7'g' 

maybe  computed  for  example  by  a  local  regression  or  by  local  optimization 
g'  maybe  based  on  given  basis  functions 


Multigrid  Schemes  as  particular  cases  of  the  suggested  Multilevel  Schemes 

Classical  Multigrid  (MG)  FAS  Schemes  in  low  dimensions  (up  to  3  usually)  on  regular 
meshes  are  obtained  as  particular  cases  of  the  suggested  multilevel  schemes. 

For  example,  for  uniform  grids,  an  operator  L  can  be  a  finite  difference  approximation  of 
a  differential  operator;  Relaxations  may  be  Gauss-Seidel  or  block  Gauss-Seidel  or 
Newton-Gauss_Seidel  relaxations,  etc. 

The  interpolations  are  based  on  local  grid  interpolations  as  in  MG. 

Local  refinement  reduces  to  performing  a  local  multilevel  scheme. 

FAS  schemes  reduce  to  MG  FAS  schemes. 


interpolations/fits  of  f 
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Observation 

Suggested  Unstructured  MG  Approach  (that  might  work  in  High  Dimensions) 

Use  Sparse  Sets  of  Points  and  Local  Expansions  for  Interpolation  and  Derivatives 


[>erivauvs  and  Inierpolaion  Use 
P1  an  Expansion(e.g.«by  Eigenfuncions) 

v' < - >  Continuous  v  < - >v‘ 

k 

u'(x,)  =  ^a,v,'(x,) 

1  r 

P2  a,u'(x,)=Xa,a,v;(x,) 


The  MG  FAS  (Full  Approximation  Scheme) 

1)  P':  F'(h')  =  /' 

2)  F';  Fhu')  =  i^=FHl‘,{u'))  +  lUi'-F'(u')) 

3)  +  («'-/;(«!.)) 


Clastic  Low  Dimension  MG 

Unstructured  MG 

Stmctursd  Mashas  of  Points 

Unstructured  spsrse  sets  of  Points 

fnterpolatlon/DerIvatIves 

Use  Meshes 

Interpolation/Derivatives 

Use  (Local)  Expansions 

Defining  Coarse  Problems 

Same 

Fine  level  exsct  solutions  transferred  to 
coarse  levels  sre  solutions  of  the  coarse 
level  problems. 

Same 

Coarse  level  solutions  do  not  change 
exsct  fine  level  solutions 

Same 

Fourier  Components 

Baals  Functions/  Elganfunctlons 

5.  A  Hierarchical  Richardson  Approach  for  Model  Extrapolation 

A  Hierarchical  Richardson  Approach  for  Model  Extrapolation  is  suggested.  Coefficients 
of  a  sequence  of  models  are  identified  by  system  identification  and  extrapolated  as  in  a 
Richardson  procedure.  Models  may  be  decomposed  into  deterministic  sub-models  (e.g., 
trends)  and  stochastic  sub-models  (e.g.,  noise).  Model  extrapolation  can  be  used  for  both 
deterministic  and  stochastic  models,  hence  for  UP.  Large  computational  savings  may  be 
obtained  by  extrapolating  a  sequence  of  coarse  model  results  to  approximate  fine  model 
results. 
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Multilevel  Extrapolation  Builds  and  Extrapolates  a  Sequence  of  Models 
Richardson  Type  Extrapolation: 

Use  a  Sequence  of  Reduced  Order  Models  for  Speedup 

Algorithm: 

1)  Find  coefficients  of  a  sequence  of  Models  by  System  Identification 

2)  Build  models  for  the  sequences  of  corresponding  coefficients  and  do  a  Sys  ID 

3)  Extrapolate  the  models  of  the  coefficients 


Build  ;  System  Idem ific lion  Model,  for  n  panicles,  QnJ  are  coefficieo. 

Eg.,  F^{x,C(n}^Y.^,{n)g,{x) 

Build  models  for  CfnJ  e.g.,  Qn)=^Djj;f 
Build  C=  lim  QnJi  e  g..  C=Oo 

UH-iO 

Build  F(x,C)  the  limit/cxiapolatcd  model  for  large  n 
E.g., 

Uncertain^  Managemem  Apply  ih  above  procedure  for  Noise  Models,  e.g.. 

Noise  Models  Data-F 
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Noise  Models _ 

Build  and  Extrapolate  a  Sequence  of  Noise  Models  by  Multilevel  Extrapolation 

Apply  the  Multilevel  Model  Extrapolation  for  Noise  Models: 

Find  the  models  F  by  System  Identification,  Define  Noise  as : 

/Voise  =  Da/a  -  F 

Build  Noise  Models  by  System  ID  using  for  the  Noise  data 


! 

1 


Extrapolated 


Extrapolated  Noise  Model 


Noise  Modeis 


37 


6.  Comparison  of  Collocation,  RS,  DS,  MC 
Applications  to  the  Kr  Phase  Diagram  Milestone 

A  Response  Surface  (RS)  Uncertainty  Quantification  Approach  is  compared  with  MC,  a 
Polynomial  Chaos  Collocation  approach,  and  with  DSAMPLE  (an  effective  quasi  MC 
technique)  on  the  Kr  Milestone  Problem.  Advantages  and  disadvantages  of  the  RS 
approach  are  discussed. 

Response  Surfaces  for  Uncertainty  Quantification 
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The  Response  Surface  (RS)  approach  of  uncertainty  quantification  (UQ)  builds 
response  surfaces  (models)  r(x)  as  approximations  of  desired  outputs  (responses) 
f(x)  as  functions  of  parameters  x  with  known  PDFs,  p(x).  In  addition  one  may  be 
interested  in  approximating  the  PDF  of  f,  see  the  diagram  below. 


Different  uncertainty  measures  of  f(x)  are  approximated  by  corresponding  measures  of 
r(x),  for  example  the  mean  or  variance  of  f  are  approximated  by  the  mean  and  variance 
of  r.  The  RS  approach  may  be  effective  when  the  evaluation  of  f(x)  is  expensive  and  a 
when  good  approximations  r  ~=  f  of  f  can  be  obtained  by  a  small  number  of  evaluations 
of  f(x).  From  the  point  of  view  of  UQ  one  is  interested  to  approximate  for  example 
mean(r)  ~=  mean(0  or  var(r)  -=  var(0-  The  main  tasks  in  the  RS  approach  are:  1 ) 
finding  a  relatively  small  number  of  sample  points  {x_i  }  and  2)  finding  a  good 
RS/interpolation  procedure  that  will  use  the  samples  {x_i  }  to  build  r  such  that  r~=  f  and 
statistical  measures  of  r  will  provide  good  estimates  of  the  desired  statistical  measures  of 
f  The  following  sections  will  discuss  an  RS  approach,  in  our  case  a  kernel  based 
interpolation  procedure  that  was  used  in  the  numerical  experiments;  domain  exploration 
approaches  for  building  sample  points  {x_i  };  techniques  for  fitting  the  pdf  of  the  output 
p(f);  and  a  comparison  of  the  RS  approach  with  Monte  Carlo  (MC),  a  Polynomial  Chaos  - 
Collocation  approach,  and  the  DSAMPLE. 


The  4  Uncertainty  Quantification  Approaches  that  have  been  compared,  MC, 
DSAMPLE,  RS,  and  Collocation,  can  be  summarized  as: 
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1)  Monte  Carlo:  perform  number  N  of  MC  evaluations  of  T(E)  for  E  generated 
by  p(E) 

mean  -=  sum(  T(Ei) )  /  N 

2)  DSample  (Quasi  MC) 
mean  -=  sum(  T(Ei) )  /  N 

3)  Response  Surface/Surrogate  Build  a  Surrogate  of  T(E),  e.g.,  using  a  basis  of 
functions,  (eigenfunctions), 

mean  -=  sum(  T(Ei)  *  p(Ei) )  /  sum(  p(Ei)  )  (use  RS  with  Importance  sampling) 

4)  Collocation  -  Polynomial  Chaos  Evaluate  T(E)  at  collocation  points  Ei,  with 
given  weights  wi 

mean  -=  sum(  T(Ei)  *  wi )  /  sum(  wi ) 

The  DSAMPLE  approach  is  a  new  quasi  MC  approach  developed  by  Igor  Mezic 
(UCSB)»  presented  in  another  chapter. 
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Uncertainty  Quantification  for  the  Krypton  Milestone 


Parameters 


MD  Simulations 


RSand  PDF 


l»m20poirw 


RS  (E.T(E))»w 
200  vm  20  potma 


RS  with  Collocation  points  reproduces  PC  efficiency  (up  to  1300  samples) 

RS  works  with  any  number  of  samples,  and  may  be  effective  in  high  dimensions  too 

Nystrom  with  probabilities  included  In  the  objective; 

RS  Optimization  (by  #  of  eigenvectors  and  sigma.  Optimization  is  not  used  in  this  example) 

The  RS  results  are  close  to  the  collocation  results, 

RS  advantage:  any  number  of  sampling  points  can  be  used  (subset  of  the  collocation  points) 

RS  disadvantage:  numencal  instability  and  difficulties  related  to  eigenvalue  computations  (time  and 
accuracy  for  diagonalization  of  large  matrices). 


Truth  is  Analytic  (VlacfillTfirJk^imit  MC(red),  Col ( Black).  RS  (blue),  DS(green) 
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RS  is  effective  for  a  small  number  of  samples,  selected  by  DS 

The  RS  Is  Optimized  (by  N  eigenvectors,  sigma,  a) 

RS  results  show  a  good  upper  trend  and  many  *‘very  good  solutions"  for  tens  of 
samples 


100  X  120  Optimal  solutions  are  generated  (sets  of  samples  from  2  to  120  are  generated  100  times, 
for  each  fixed  set  of  samples  the  RS  is  optimized  for  N  of  eigenvectors,  sigma,  a) 

The  mean  and  variance  of  the  optimal  RS  are  selected  (and  a  blue  point  of  the  solution  Is  shown,) 


MEAN 


10’  1(f  1C?  ^(f  1C?  icf  1C?  icf  icf  lo^io” 


VARIANCE 


New  England  Power  Grid  -  State  Uncertainty  Propagation 

The  propagation  of  state  PDFs  in  Dynamical  Systems  by  sums  of  Gaussians  can  be 
performed  as  in  Extended  Kalman  Filter  approaches  (see  for  example:  Uncertainty 
Propagation  for  Nonlinear  Dynamical  Systems  using  Gaussian  Mixture  Models 
Gabriel  Terejanu  ,  Puneet  Singlat ,  Tarunraj  Singht ,  Peter  D.  Scott  1  AIAA  Guidance, 
Navigation  and  Control  Conference  and  Exhibit  AIAA  2(X)8-7472  ;  18-21  August  2(X)8, 
Honolulu,  Hawaii)  by  the  following  algorithm: 


-  The  initial  state  x(0)  of  a  dynamical  system  x(t+l)=f(x(t))  has  given 
PDF(x(0)) 

Propagate  PDF(x(t))  as  an  approximation  of  a  Sum  of  Gaussians: 

Gi(x(t))  =  wi  *  fi  *  exp  ( -  (x-mi)’  Si^(-l)  (x-mi)  /  2  )  ;  mi:  mean,  wi:  amplitude; 
Si:  covariance  matrix,  fi  =  1  /  (  det(Si  )'^(l/2)  *  (2pi)'^n/2  ) 
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Propagate  means  mi  by  the  dynamical  system  x_mean(t+l)  =  f(x_mean(t)) 

Propagate  covariance  (width)  using  Jacobians  (at  means): 

Si(t+1)=  (Df/Dx)  Si(t)  (Df/Dx)’ 

Keep  the  weights  wi  constant  or  approximate  them  by  an  optimization 


The  approach  was  applied  to  the  New  England  Power  Grid  model  (as  in  the  paper:  Global 
Swing  Instability  of  Multimachine  Power  Systems;  Yoshihtko  Susuki,  Igor  Mezi'c, 
Takashi  Hikihara;  Proceedings  of  the  47th  IEEE  Conference  on  Decision  and  Control 
Cancun,  Mexico,  Dec.  9-11, 2008)  as  shown  in  the  figures  below. 


New  England  Power  Grid 


Global  Swing  Instability  of  Multimachina  Power  Systems 
Yoshihiko  Susuki,  Igor  K^zi'c,  Takashi  Hikihara 
Proceedings  of  the  47th  IEEE  Conference  on  Decision  and  Control 
Cancun,  Mexico,  Dec.  9-11, 2006 
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New  England  Power  Grid  -  State  Uncertainty  Propagation 

Propagate  the  PDF  of  the  State  of  the  system,  PDF(x)  =  Sum  of  Gaussians 

x;  18  states;  propagation  of  1  Gaussian  with  initial  mean  and  Covariance 
Implicit  ODE  simulator  OOE1 5s, 


Propagation  of 

norm  of  mean  of  Gaussian 


Propagation  of  Norm  of 
Covariance  Matrix  of  Gaussian 


2  sec 


200  sec  MUti. 


The  System  tends  to  behave  more  and  more  deterministic  (due  to  stability) 

4 


PDF  merging  for  the  New  England  Power  Grid 

A  combination  of  the  proposed  PDF  merging  approach  with  the  propagation  of  sums  of 
Gaussians  approach  is  suggested  in  the  next  scheme.  The  large  number  of  input  PDFs 
may  come  from  the  loads  of  the  grid,  for  example  PDFs  of  grid  loads  of  different  cities. 

In  this  case,  large  computational  savings  can  be  obtained  by  an  early  merging  of  the 
many  input  PDFs  into  state  PDFs  (there  are  only  18  states),  that  are  further  approximated 
and  propagated  by  sums  of  Gaussians  as  illustrated  above.  The  first  half  of  the  scheme, 
the  simulation  for  140  input  PDFs  and  generation  of  the  1 8  state  PDFs  was  performed 
using  DSample.  The  second  half,  although  not  computed,  can  be  performed  with  the  sum 
of  Gaussians  code  for  the  NE  Grid  as  illustrated  above.  The  computational  time  savings 
can  be  estimated  by  the  formulas  provided  in  section  I .  For  example,  assuming  that  about 
lO^n  samples  are  needed  for  the  initial  simulations  using  the  input  PDFs  to  be  merged 
into  state  PDFs,  and  that  about  10^3  simulations  are  needed  for  the  propagation  of  state 
PDFs,  and  that  the  simulation  time  used  for  merging  the  PDFs  is  short  compared  to  the 
total  timulation  time,  then  the  time  savings  would  be  of  order  IO^(n-3). 
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PDF  merging  for  the  New  England  Power  Grid 
Combination  of  DS  with  State  PDF  Propagation 


□SAMPLE  for 
1 40  parameters 
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B.3  Approximate  solutions  for  decentralized  detection/estimation 
problems 
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Approximate  solutions  for  decentralized 
detection/estimation  problems 

DyNARUM  report 


Jong-Han  Kim 


1  Introduction 

We  consider  a  class  of  decentralized  detect  ion/esti  mat  ion  problems,  where  multiple  agents 
seek  for  the  optimal  decision  rule  or  estimator  for  the  global  objective  function,  i.c., 
the  objective  function  depends  on  every  agent’s  performance.  E^ch  agent  uses  its  own 
measurements,  and  we  especially  consider  cases  where  the  communication  between  agents 
is  impossible.  In  this  specific  setting,  the  optimization  problem  is  formulated  and  solved 
a  priori  by  the  mission  center,  then  the  optimal  rule  for  each  agent  is  implemented  on 
the  on-board  softwares  before  the  search  mission  begins.  As  mentioned,  no  inter-agent 
communication  during  the  mission  is  allowed,  therefore  the  only  information  available  to 
an  agent  is  the  measurements  taken  by  itself. 


2  Decentralized  detection 

We  describe  the  2-agent  decentralized  detection  problem  as  follows. 


L  X  L  A/  X  A/ 

minimize  E  E  Cyu^yu 

V  « 

subject  to  K  —  ^ 

=  1 

gB^xA/  for  n=  1,2 

where  C  describes  some  cost  definition,  and  represents  the  nth  agent’s  decision 

matrix.  The  operator  (S)  represents  the  Kronecker  product  of  two  matrices. 

Note  that  solving  this  problem  is  hard  because  of  the  nonconvexity  due  to  the  bilinear 
function  and  the  Boolean  constraints.  So  we  explore  a  relaxation  technique  that  solves 
the  relevant  problem  easily,  providing  an  acceptable  bounds  for  the  optimal  cost. 

Defining  the  new  variables  x  =  •  '^kjf  gj  •  •  •  gjf  l]^  and  Z  =  where 

A"^D  =  [A:i  •••  A:a/]  and  A"^^l  =  [pi  *•*  9Sf]^  problem  is  transformed  to  a 

semidefinite  programming  (SDP)  with  the  rank —I  nonconvex  constraint.  Dropping  the 
rank— 1  constraint  yields  the  following  SDP  relaxation. 


1 
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subject  to 


where 


Z12 

ZiM 

Z21 

Z22  ■  ■ 

.^Ll 

Zl2  '  • 

LxL  M  xAf 

i:  i:  c.. 

V  u 


AZb  =  1,  AZ  =  \b^Z,  b^Zb  =  1 
diag(Z)  =  Z6,  0  :<  diag(Z)  :<  1 
^ij  ^  0,  Zjj  <  Ziiy  Zij  <  Zjj 

Zii  ”l-  Zjj  —  Zij  <  1 

Z  V  0 

I 


for  all  iy  jy  j 


A  = 


LxL 

OlxL 


^LxL 
pLxL 

-^LA/  +  2.(j-l)L+t 


Olxl 

^LxL 


Olxl 

^  LxL 

^L(A/  +  l)  +  l,(j-l)L+i 
^L(A/  +  l)+2.(j-l)L+t 


6  = 


02LAf  xl 
1 


.2'LA/+L,(>-l)L+i  -^L(A/  +  l)+L,(j-l)L+» 

for  i  ^  j  ^ 

Note  that  the  SDP  relaxation  provides  a  lower  bound. 

•^SDP  ^ 


-^L(2A/-l)-|-l,(j-l)L  +  i 
-^L(2Af-l)+2,(j-l)L+t 

‘^L(2Af-l)  +  L,0-l)L  +  i_ 


If,  fortunately,  the  relaxed  problem  finds  the  optimum,  Zg^p,  such  that  rank(Z5£,p)  = 
1,  it  suffices  to  prove  that  the  above  bound  is  tight  and  the  relaxed  optimum  is  equal  to 
the  true  optimum,  i.e.,  Jsdp  ~ 

Though  the  condition  or  the  proof  for  the  zero  optimality  gap  is  not  known  yet,  it  turns 
out  that  the  relaxed  problem  successfully  finds  the  true  optimum  in  the  major  number  of 
random  numerical  simulations.  However  it  sometimes  fails  to  find  the  rank— 1  solution, 
resulting  in  a  solution  infeasible  to  the  original  problem.  The  following  heuristic  approach 
will  help  to  find  a  suboptimal  rank— 1  solution  for  such  cases. 

Consider  the  following  SDP. 

minimize  trace(Z^VP) 

subject  to  0  :<  IP  :<  / 

trace(VV' )  =  2LM 


where  the  optimal  value  is  equal  to  the  sum  of  all  the  eigenvalues  except  the  largest  one. 

Note  that  IP*  is  obtained  explicitly  as  IP*  =  where  (/  =  [u2  W3  *  •  •  W2LAf]- 

Uk  represents  the  Arth  singular  vector  of  Z.  Here,  we  are  assuming  that  Z  is  known 
(from  the  previous  iteration).  Since  Z  orthogonal  to  such  fixed  IP  attains  the  minimum, 
augmenting  this  to  the  relaxation  problem  tends  to  pull  the  optimum  to  the  set  of  rank  —  1 
matrices. 

Now  augmenting  the  relaxed  problem^with  this  heuristic,  we  get  the  augmented  SDP 
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relaxation. 


Z\2  •  • 

Z\M 

LxL  MxM 

Z21 

Z22  •  • 

Z2M 

minimize 

V  « 

Zli 

Zl2  ■  • 

subject  to 

Zgp 

W  is  obtained  from. 

minimize 

tTace(ZI,J„,W) 

subject  to 

0  :<  VV'  r<  / 

trace(lP)  =  2LM 

+  trace(Z^lV') 


where  J'  is  the  convex  feasible  set  of  the  SDP  relaxation  problem. 

Solving  the  augmented  relaxation  iteratively  will  give  a  rank-1  solution,  hopefully  find¬ 
ing  the  true  optimum  of  the  original  problem,  or  at  least  providing  a  reasonable  upper 
bound  of  the  optimal  cost. 


•^SDP  ^  S.  Jaug 

A  large  number  of  numerical  simulations  are  conducted  in  order  to  investigate  the 
performance  of  the  SDP  relaxation  and  the  augmented  SDP  relaxation.  The  random  cost 
matrices  with  L  =  4,  and  A/  =  4  are  constructed  by  several  different  rules.  Although 
the  series  of  random  simulations  does  not  represent  all  the  posibilities  or  can  not  prove 
anything,  it  provides  an  idea  of  how  well  the  proposed  schemes  is  going  to  perform  in 
average  sense.  In  summary,  it  was  observed  that  in  approximately  more  than  98%  of  the 
total  proper  trials,  the  SDP  relaxation  /  augmented  relaxation  successfully  found  the  true 
optimum.  Even  for  the  rest  cases  where  it  failed,  the  provided  gap  was  fairly  acceptable 
(about  several  %  of  the  optimum  value). 

3  Decentralized  static  estimation 

Decentralized  static  estimation  problem  described  here  is  a  natural  counterpart  of  the 
decentralized  detection  case.  Recall  that  the  decision  function  of  the  detection  problem 
maps  a  discrete  measurement  to  a  discrete  decision  variable,  t.e.,  7n(j/n)  :  {li***»^}  — ► 

,A/}. 

In  the  decentralized  static  estimation  problem,  each  agent  takes  a  noisy  measurement 
vector  y„  G  and  the  estimator  function  'yniVn)  returns  the  optimal  estimate  u„  G 
minimizing  some  global  objective  function. 

VVe  consider  the  2-agent  decentralized  static  estimation  problem.  For  a  static  state 

G  with  its  a  priori  statistics  known  as  x  'v/  A/"(/i,  P),  the  mesurements  y\  G  and 
2/2  €  R"*^i  corrupted  by  the  independent  noise  Wi  ~  Ar(0,  A^i),  are  given  to  each  agent 
respectively,  t.e., 

yi  =  Aix-f  u’l 
2/2  =  A2X  +  W2 

Then  the  optimal  estimate  is  found  by, 

minimize  Ec(ui,U25^) 
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where  =  IniVn)  the  estimator  of  nth  agent. 

If  the  function  c(ui ,  U2,  x)  is  convex  quadrataic.  it  is  known  that  there  exists  the  optimal 
7„  which  is  linear.  With  the  reasonable  choice  of  the  cost  function  c(ui,U2,x)  =  ||ui  — 
x||^  +  11^2  -  x|p  -h  (J^||ui  —  U2||^,  the  optimal  estimator  u*  =  7*j(yu)  =  ^nVn  +  ^re 
found  from  the  following  relations. 


-5'^AxPAl 

0 

0 

’Kf 

'ArP 

-S^AiPAj 

(\+5'^){A2PAI  +  N2) 

0 

0 

Kl 

A2P 

-g^y'Ai 

1+5^ 

-<52 

T 

{\+^)yAi 

-<52 

1+52 

A. 

T 

1  ^  J 

For  the  simplest  case  with  5  =  0,  the  problem  is  decomposed  into  two  independent 
problems.  Then  the  solution  corresponds  to  the  well-known  update  formula. 

K„  =  PAl{A„PAl  +  N„y^ 

or  Un  =  /t  +  PA^(A„PA^  +  Nny\y„  -  An^) 


The  optimal  solution  is  also  found  from  the  following. 

ii; 

TA'i 


minimize 


subject  to 


C  -h  DKB 
K  = 


0 

0 


where  z  = 

C  = 


,iv  = 


Nx  0 
0  ^2 


,B  = 


{C^DKB) 

Ax  I  0 
A2  0  I 


“h  Dz  =  0 


■/ 

0  " 

pl/2 

0 

0 

0  0 

.  D  = 

0 

SI 

I 

-SI 

,  B  =  B 

0 

,  and  C  =  C 


pl/2  0 
0  iVV2 


This  is  a  simple  convex  problem,  whose  optimal  solution  should  be  identical  to  the 
explicit  result  in  the  previous  section. 


4  Next  directions 

The  relaxed  decentralized  detection  problem,  which  provides  a  lower  bound  of  the  optimal 
cost,  was  studied.  Then  the  relaxed  problem  is  augmented  and  iterated  to  find  a  sub- 
optimal  primal  feasible  solution.  A  series  of  random  simulations  demonstrated  that  the 
combination  finds  the  true  optimum  in  most  cases,  or  at  least  provide  a  satisfactorily  tight 
gap.  We  are  interested  in  discovering  the  conditions  under  which  the  relaxed  problem  or 
the  augmented  iteration  finds  the  exact  optimum  with  zero  bounds,  and  developing  the 
proofs  for  such  conditions. 

As  a  natural  extension  of  the  decentralized  detection  problem,  we  formulated  basic 
frameworks  for  the  decentralized  static  estimation  problem.  Under  the  convex  quadratic 
cost  and  the  affine  estimator  assumptions,  we  showed  that  the  optimum  is  easily  found. 
An  interesting  and  realistic  extension  is  the  decentralized  dynamic  estimation  problem 
(with  limited  inter-agent  communication  assumption),  which  we  are  currently  putting  our 
efforts  to. 
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Reduced  Order  Representations  for  Efficient  Computation 

Laurent  Lessard 
Stanford  University 


Motivation 

Consider  a  large  distributed  estimation  problem  with  N  agents  that  must  estimate  each  other’s  positions. 
The  optimal  estimate  can  depend  on  global  information  structures,  such  as  a  densely  populated  NxN 
covariance  matrix. 

Taking  a  simple  example:  we  would  like  to  estimate  jc/,  X2,  ... .  jca^,  and  each  agent  takes  a  single 
measurement  =  jc*  +  v*,  where  v  is  white  noise  with  covariance  and  the  vector  jc  has  a  normally 
distributed  prior  with  covariance  matrix  C. 

In  this  case,  the  optimal  estimate  is  the  vector:  jCopi  =  C  (C+o^/)'*  y.  Our  goal  is  to  compute  such 
quantities  efficiently,  in  0(N).  This  means  that  the  total  computation  cost  is  proportional  to  the  number 
of  agents;  adding  more  agents  does  not  change  the  computational  burden  of  each  agent.  Note  that  if  we 
can  also  distribute  the  computation  evenly  amongst  the  agents  and  execute  it  in  parallel,  the  estimation 
requires  a  constant  0(1 )  cost. 

In  order  to  achieve  this,  we  perform  certain  basic  computations.  Namely,  if  w  is  an  arbitrary  vector  of 
length  N,  we  should  know  how  to  compute  matrix  multiplications  and  inverses:  computing  Cvv  or  C*vv 
in  0{N).  We  consider  problems  in  which  C  has  a  high  degree  of  symmetry.  Namely,  it  is  shift-invariant 
(Toeplitz  or  circulant).  We  will  show  how  to  factorize  such  systems  in  a  way  that  yields  the  desired 
computational  speedup. 

Sparse  Matrices 

An  NxN  sparse  matrix  is  a  matrix  that  contains  0{N)  nonzero  elements.  If  P  is  sparse,  Pw  can  be 
computed  in  0{N).  Also,  for  certain  classes  of  P,  P'^w  can  be  computed  in  0(N)  as  well.  This  is 
typically  done  with  a  multigrid  method  using  a  small  fixed  number  of  iterations.  Such  algorithms  can 
be  implemented  in  a  distributed  fashion.  Note  that  P'*  is  not  sparse  in  general,  and  that  iterative  solvers 
never  compute  explicitly. 

Circulant  Factorization 

Circulant  matrices  are  constant  along  each  diagonal,  and  the  diagonals  wrap  around.  In  other  words, 
each  column  of  a  circulant  matrix  can  be  obtained  by  taking  the  previous  column  and  applying  a 
circular  shift  of  one.  All  circulant  matrices  are  diagonalized  by  the  discrete  Fourier  transform  (DFT) 
matrix. 

If  the  eigenvalues  are  generated  by  a  rational  function: 

.  Po  +  Pi-A- +  4" 

=  - 

7(1  +  7i~*-  - 1-  7»n! 
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where  Zk  =  exp(  2mkJN )  are  evenly  distributed  on  the  unit  circle,  we  can  factor  C  as: 


Pi) 
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q\ 

•:  Pi) 
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•••  /y 
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...  7o 

- - "" - - 
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where  P  and  Q  are  sparse  banded  circulant  matrices.  The  factorization  C-PQ^  is  very  beneficial, 
because  it  allows  us  to  compute  Cw  by  first  solving  w-Qz,  and  then  computing  >’=Pe.  The  total  cost  is 
0(N).  We  can  compute  C'^w  in  0(N)  similarly.  Note  that  without  this  factorization,  the  cost  would  be 
0{N\ogN),  using  an  FFT  method. 

This  factorization  is  only  exact  when  we  are  dealing  with  a  rational  eigenvalue  distribution,  as 
explained  above.  In  the  general  case,  we  will  seek  an  approximate  factorization  C^PQ’\  For  some 
classes  of  problems,  we  can  find  an  optimal  approximate  factorization  for  C  in  the  sense  that  we  can 
trade  off  sparsity  (width  of  the  bands  in  P  and  Q)  with  solution  accuracy  (the  error  ICw  -  PQ'\v] ). 

Example:  wavefront  sensing 


In  wavefront  sensing  for  adaptive  optics,  one  encounters  the  von  Kdiman  power  spectral  density  (PSD) 
for  atmospheric  turbulence  in  1-D: 


1 

(F4-  1)V3 


The  correlation  between  two  points  separated  by  a  distance  r  can  be  computed  by  integrating  and 
sampling  the  PSD.  If  we  have  an  array  of  evenly  spaced  detectors  with  spacing  Ar,  the  pairwise 
coefficients  can  be  assembled  in  a  Toeplitz  matrix  C.  We  would  like  to  compute  Cw  efficiently  for  any 
w,  but  C’s  PSD  is  irrational  so  the  eigenvalues  distribution  is  as  well.  Using  a  Principal  Axis 
optimization  routine,  we  computed  some  low-order  approximations  for  the  eigenvalue  distribution  that 
minimize  the  mean-squared  approximation  error  under  white-noise  inputs.  In  other  words,  we 
minimized  EICw  -  PQ'V’P.  See  below  for  a  figure  comparing  the  PSD  error  in  various  rational 
approximations: 
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It  turns  out  that  in  the  limit  of  large  N,  this  is  equivalent  to  minimizing  the  Li  error  in  the  PSD’s 
approximation.  For  example,  the  factorization  we  found  in  the  (1,2)  case  using  zfr=0.2  is: 
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where  p  =  [0.100126  -0.0301038]  and  q  =  [1 .0000000  -0.650885  0.15151 8].  This  case  leads  to  an 
average  error  of  about  2.5%  in  each  component  of  the  product  Cw.  This  error  does  not  depend  on  N  in 
the  limit,  and  in  fact,  the  same  factorization  can  be  used  for  larger  N  (keep  the  same  p  and  q,  and  extend 
the  matrices). 


Using  such  a  factorization  is  always  0(N).  If  we  wanted  to  do  better  than  2.5%,  we  could  find  a  higher- 
order  approximation.  This  would  increase  the  amount  of  computation  each  agent  has  to  do,  but  the 
computation  required  would  always  be  independent  of  the  number  of  agents,  N. 

Conclusion 

We  showed  that  a  shift-in  variant  matrix  can  be  factored  as  C=PQ“*  using  sparse  matrices  P  and  Q  if  the 
eigenvalues  of  C  are  rational  functions  on  the  unit  circle.  Such  a  factorization  leads  to  fast  distributed 
computation  without  any  performance  penalty. 

If  the  eigenvalues  are  not  rationally  distributed,  we  can  look  for  a  rational  approximation.  We  showed 
an  example  of  a  distributed  estimation  problem  where  we  could  find  optimal  rational  approximations  of 
a  given  order.  By  varying  the  chosen  order,  we  obtain  a  family  of  0(N)  solvers  that  trade  off 
computational  accuracy  and  mean-squared  error  performance. 
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Abstract 

Contemporary  systems  biology  presents  science  and  engineering  with  many  new  challenges. 
Finding  effective  ways  to  study  large  networks  of  interacting  components  is  one  of  the  key  tasks  of 
this  growing  discipline.  Because  of  their  numerous  applications  in  drug  target  design  and  bio  reactor 
technology,  metabolic  networks  are  of  special  interest.  These  networks  are  usually  modeled  as 
dynamical  systems  on  graphs.  This  approach  allows  to  accurately  simulate  the  network  behavior  in 
different  conditions  by  solving  large  systems  of  differential  equations  with  an  even  larger  number  of 
parameters.  However  as  we  attempt  to  analyze  larger  networks,  the  growing  number  of  equations 
complicates  numerical  analysis.  In  the  present  paper  we  employ  a  horizontal-vertical  decomposition 
(HVD)  method  that  helps  to  analyze  dynamical  systems  on  graphs  that  have  very  large  dimensions. 
We  illustrate  the  HVD  decomposition  on  metabolic  networks  of  several  single  cell  organisms.  Using 
HVD  decomposition  we  identify  modular  structure  and  polarity  in  the  network,  which  in  turn  allows 
us  to  reduce  the  dimension  of  the  problem. 


1  Introduction 

In  modern  biology  and  medicine  there  is  a  great  need  of  understanding  biological  processes  integrated 
in  their  system  environment.  The  study  of  complex  interactions  in  biological  networks  has  grown  into 
an  independent  discipline  of  systems  biology  important  parts  of  which  are  mathematical  modeling  and 
dynamic  simulation  [18]. 

Even  though  presently  the  amount  of  experimental  data  is  growing  fast,  quantitative  measurements 
of  many  cellular  components  are  still  unavailable.  Shortage  of  data  in  combination  with  biological  com¬ 
plexity  makes  it  very  challenging  to  apply  standard  engineering  methods  for  modeling,  simulation  and 
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mathematical  analysis  of  biochemical  networks  [12].  The  existing  biochemical  data  measurements 
usually  come  from  different  experimental  settings  and  are  described  on  different  levels  of  information 
quality,  besides  for  most  processes,  quantitative  information  on  reaction  rates  and  molecular  concen¬ 
trations  is  not  directly  accessible  in  vivo.  [12].  In  cases  where  the  mechanistic  details  are  unclear,  it  is 
necessary  to  fill  in  the  gaps  by  suggesting  simple  mechanisms  without  having  any  kinetic  parameters 
available  [10].  If  the  dynamic  behavior  of  a  system  is  highly  dependent  on  the  value  of  (some  of)  the 
parameters,  then  accurate  and  reliable  quantification  of  the  parameters  is  essential  for  the  develop¬ 
ment  of  predictive  models  [19].  Therefore  being  able  to  predict  how  values  of  certain  variables  of  the 
biological  system  affect  other  variables  is  very  important. 

In  [11]  Mezic  introduced  a  horizontal  -  vertical  decomposition  method  (HVD)  to  analyze  asymp¬ 
totic  behavior  and  uncertainty  propagation  in  nonlinear  dynamical  systems  on  graphs.  We  illustrate 
the  HVD  decomposition  on  metabolic  networks  of  several  single  cell  organisms  and  assume  that  the 
dynamics  of  the  system  is  described  by  a  system  of  non-linear  ordinary  differential  equation  (ODE).  In 
case  of  metabolic  networks,  the  variables  may  represent,  for  instance,  concentrations  of  the  metabo¬ 
lites,  and  the  equations  describe  how  changes  in  concentrations  depend  on  each  other.  Modelling 
metabolic  networks  as  dynamical  systems  has  many  advantages.  The  dynamical  system  approach  is 
very  convenient  to  describe  features  like  feedforward  and  feedback  loops,  and  to  study  concepts  of 
stability,  robustness,  homeostasis  and  adaptation.  Homeostasis  is  one  of  the  most  typical  properties  of 
highly  complex  open  systems.  In  dynamical  system  setting  it  means  that  when  a  disturbance  occurs, 
interdependent  regulation  mechanisms  make  the  system  converge  to  a  dynamic  equilibrium.  HVD 
allows  us  to  extract  valuable  information  on  asymptotic  dynamics  of  metabolic  networks  using  its 
structure  without  precise  estimates  of  the  differential  equations. 

Large  genome  sequencing  projects  made  possible  the  reconstruction  of  complete  metabolic  and 
signaling  networks  for  many  organisms.  In  [14]  Barabasi  and  his  coworkers  performed  quantitative 
analysis  of  the  topological  properties  of  complete  metabolic  networks  of  42  organisms,  25  of  which 
were  fully  sequenced.  These  metabolic  networks  were  reconstructed  based  on  data  deposited  in  WIT 
database,  see  [13].  We  analyze  qualitative  features  of  metabolic  networks  from  [14],  using  essentially 
the  same  model  with  the  only  difference  that  we  do  not  represent  the  temporary  substrate-enzyme 
complexes  in  the  network. 
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2  Horizontal-vertical  decomposition 

Metabolic  networks  are  usually  modelled  as  nonlinear  dynamical  systems  on  graphs.  In  our  model 
vertices  of  the  graph  represent  the  metabolites  (substances  produced  or  used  during  metabolism) 
participating  in  reactions.  Each  reaction  is  catalyzed  by  a  unique  enzyme  wdiich  w^e  do  not  represent 
by  a  vertex.  For  each  chemical  reaction  that  takes  place  in  the  metabolism,  the  vertices  representing 
reaetants  are  eonneeted  to  the  produets  of  the  reaction.  For  example,  if  we  have  a  re^LCtion  Vi  -f  V2  = 
Vs -\-V4  It  will  be  represented  by  the  graph  shown  in  figure  2.  The  eonneeting  edges  are  directed  and 
always  point  to  the  product  of  the  reaction.  We  assume  that  there  are  no  multiple  edges  in  the  graph. 
It  means  that  there  will  be  one  edge  going  from  vertex  A  to  B  even  if  there  is  more  than  one  reaction 
involving  reactant  A  and  producing  reactant  B,  As  an  example  of  such  model,  the  graph  shown  in 
figure  1  is  the  metabolic  network  model  of  Chlamydia  pneumoniae  (C.  pneumoniae).  This  graph  has 
187  vertices  (the  number  of  substances  in  the  network)  and  652  edges. 

We  assume  that  to  each  vertex  of  the  graph  belongs  a  variable  which  is  the  concentration  of 
that  metabolite  and  an  ODE  that  describes  how  the  change  in  concentration  of  a  given  metabolite 
depends  on  other  metabolite  concentrations.  It  is  elear  that  a  system  of  the  187  differential  equations 
corresponding  to  this  graph  can  be  very  complicated.  However,  using  HVD  decomposition  we  can 
reduce  the  dimension  of  the  problem  by  revealing  modular  structure  and  polarity  of  the  network.  In 
[11]  in  Theorem  13  it  was  proved  that 

Theorem  2.1  A  dynamical  system  can  be  decomposed  into  k  vertical  levels,  such  that  each  higher 
level  is  driven  by  the  dynamics  of  levels  below.  Every  horizontal  level  i  E  can  be  decomposed 

into  mi  sets,  which  have  dynamics  independent  of  each  other. 

This  decomposition  is  called  horizontal- vertical  decomposition  (HVD)  of  the  system.  The  proof 
of  theorem  2.1  is  based  on  repeatedly  separating  sets  of  variables  with  isolated  dynamics  using  its 
Jacobian  (see  [11]).  We  assume  that  at  each  step  we  separate  the  maximal  number  of  such  sets,  and 
we  will  regard  this  unique  horizontal- vertical  arrangement  as  the  HVD  decomposition. 

Note,  that  in  general  the  number  of  layers  k  in  decomposition  of  this  type  may  vary.  For  example, 
we  could  define  a  separate  layer  for  each  set  having  isolated  dynamics  in  which  case  the  number  of 
layers  w'ould  be  equal  to  ]Ci=i  •  However,  for  a  fixed  set  of  parameters,  the  partition  of  the  set  of 
variables  into  sets  with  isolated  dynamics  is  unique  and  does  not  depend  on  the  number  of  layers  in 
the  decomposition. 

The  adjacency  matrix  of  a  directed  graph  G  on  n  vertices  {ci,t?2,  is  a  n  x  n  matrix  A  with 
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Figure  1:  Metabolic  network  of  C.  pneumoniae 


Figure  2:  Graph  with  adjacency  matrix  A 
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entry  a,-j  equal  to  1  if  VjVj  is  an  edge  and  0  otherwise.  In  figure  2  is  an  example  of  a  directed  graph 
that  has  the  following  adjacency  matrix. 


f  0 
0 
0 
0 


0  1  1  \ 
0  1  1 
0  0  0 
0  0  0  / 


The  following  two  important  observations  for  our  metabolic  network  model  follow  from  [11]. 


Observation  2.2  Assume  that  a  metabolic  network  is  described  by  a  directed  graph  with  an  n  x  n 
adjacency  matrix  A  and  a  system  of  n  differential  equations  with  Jacobian  J.  An  entry  of  the  transpose 
of  the  adjacency  matrix  A^  is  non-zero  if  and  only  if  the  corresponding  entry  of  J  is  non-zero. 


A  directed  graph  is  called  strongly  connected  if  for  every  pair  of  vertices  u  and  v  there  is  a 
directed  path  from  u  to  v  and  from  v  to  u.  The  strongly  connected  components  are  the  maximal 
strongly  connected  subgraphs  of  a  directed  graph.  To  shorten  the  description,  we  will  use  the  term 
strong  component  instead  of  strongly  connected  component.  We  will  use  notation  SC  for  a  strong 
component.  Strong  components  form  a  partition  of  the  vertex  set  of  the  graph. 


Corollary  2.3  Assume  that  a  metabolic  network  is  described  by  a  directed  graph  G  with  an  n  x  n 
adjacency  matrix  A  and  a  system  of  n  differential  equations  with  Jacobian  J.  Then  for  every  level  i  G 
{1,...,A:}  of  the  HVD  the  subsets  mi  with  independent  dynamics  described  in  Theorem  2.1  correspond 
to  strong  components  of  G. 

The  proof  of  the  following  lemma  is  straightforward,  see  [5]. 

Lemma  2.4  If  each  strong  component  is  contracted  to  a  single  vertex,  the  resulting  graph  is  cycle-free, 
i.e.  is  a  directed  acyclic  graph. 

Assume  that  we  have  a  metabolic  network  described  by  a  directed  graph  G.  From  Corollary  2.3 
and  Lemma  2.4  it  follows  that  if  we  contract  each  strong  component  of  G  to  a  single  vertex  ,  the 
resulting  graph  SC{G)  is  acyclic.  Each  vertex  of  SC{G)  corresponds  to  a  set  with  isolated  dynamics 
in  the  HVD  decomposition.  To  complete  the  HVD  decomposition  of  the  metabolic  network,  we  have 
to  assign  each  vertex  of  SC{G)  to  a  level.  We  call  a  vertex  sink,  if  it  has  only  incoming  or  in-edges 
edges  (but  no  out-edges).  We  call  a  vertex  source  if  it  has  only  out-edges  edges  (but  no  in-edges). 
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Figure  3:  HVD  graph  of  the  metabolic  network  of  C.  pneumoniae 

First,  we  find  the  set  of  all  the  sinks  in  SC{G)^  S\  and  place  them  in  the  top  level.  The  vertices 
of  S\  correspond  to  the  sets  with  isolated  dynamics  that  we  separate  first,  according  to  the  proof  of 
Theorem  2.1.  Next,  we  delete  the  set  of  vertices  S\  from  5C(G),  and  place  the  set  of  all  the  sinks  S2 
of  SC{G)  —  Si  in  the  next  level.  Then,  wc  delete  the  sets  of  vertices  S\  and  52  from  5C(G),  and  place 
the  set  of  all  the  sinks  53  in  the  resulting  graph  5C(G)  —  5i  —  52  in  the  next  level...  Continuing  in 
this  manner  we  obtain  the  HVD  graph  G*  of  the  metabolic  network.  All  the  sinks  of  5G(G)  will  be 
in  the  top  level  of  G*  and  all  the  sources  of  5G(G)  will  be  in  the  bottom  level  of  G*.  To  summarize 
the  above  procedure,  finding  the  partition  of  the  dynamic  model  of  the  metabolic  network  into  sets 
with  isolated  dyneunics  is  equivalent  to  finding  the  strong  components  of  the  corresponding  graph  G, 
which  is  a  well  known  problem  in  graph  theory.  HVD  decomposition  G*  of  a  dynamical  system  with 
graph  G  can  be  found  in  3  steps: 

1.  Find  strong  components  of  G 

2.  Contract  each  strong  component  to  a  single  vertex  to  find  5G(G) 

3.  Assign  vertices  of  5G(G)  to  levels,  to  obtain  the  HVD  graph  G* 

If  Ci  is  a  concentration  of  interest,  we  can  find  the  trajectory  of  Ci  considering  the  system  of 
dynamical  equations  corresponding  to  all  variables  that  are  below  Cj  in  the  HVD  decomposition  and 
can  reach  c,  via  directed  path.  Using  the  metabolic  network  of  Chlamydia  pneumoniae  as  an  example, 
let  us  demonstrate  how  this  approach  can  result  in  a  considerable  reduction  in  the  dimension  of  this 
problem.  In  figure  3  there  is  a  picture  of  the  HVD  graph  of  the  metabolic  network  of  Chlamydia 
pneumoniae.  There  are  52  vertices  in  this  graph  and  each  vertex  represents  a  strong  component. 
There  are  9  single- vertex  components  that  send  an  edge  to  component  A  Ac,  they  are  schematically 
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represented  by  a  single  vertex.  Also,  there  are  arc  9  single- vertex  components  each  of  that  receive 
an  edge  from  component  AAc,  they  are  also  represented  by  a  single  vertex  on  the  picture  of  the 
HVD  graph.  Wc  will  call  the  size  of  a  strong  component  the  number  of  vertices  in  it.  There  is  one 
strong  component  of  size  136  (AAc,  shown  in  figure  3  in  red),  all  other  strong  components  consist  of 
a  single  vertex.  Assume,  that  the  concentration  Cj  that  is  of  interest  to  us  corresponds,  for  instance, 
to  the  single- vert  ex  component  A  Ad.  In  this  case  it  is  sufficient  to  consider  the  system  of  equations 
corresponding  to  A  Ad,  AAe,  AAv  and  AAp  to  find  trajectory  of  q.  This  means  that  instead  of  having 
to  solve  a  system  of  187  differential  equations  as  defined  in  the  original  problem  we  reduced  finding 
the  trajectory  of  Ci  to  solving  a  system  of  4  differential  equations.  However,  if  Cj  is  in  the  136  vertex 
component  AAc  we  still  have  to  consider  a  system  of  149  equations  (these  correspond  to  vertices  in 
components  AAc,  AAu,  AAp,  AAz,  AAv,  the  9  single- vertex  components  represented  by  vertex  9SCCs 
and  the  component  AAc  itself).  It  is  clear  from  our  example  that  in  order  to  substantially  reduce  the 
dimension  wc  have  to  also  decompose  the  giant  strong  component  itself.  In  the  next  chapter  we  will 
propose  a  way  to  do  it. 

There  are  several  algorithms  that  compute  strong  components  of  a  graph.  Tarjan’s  algorithm  [5]  is 
one  of  the  most  favored  in  practice.  It  finds  strong  components  in  time  linear  in  the  size  of  the  graph. 

3  The  giant  strongly  connected  component 

Each  of  42  metabolic  networks  that  we  tested,  had  one  very  large  strong  component,  containing  about 
84%  of  all  vertices  and  many  single-vertex  components  with  occasional  components  of  size  2  or  3.  We 
will  refer  to  the  largest  strong  component  of  the  metabolic  networks  as  the  giant  strongly  connected 
component,  denoted  GSC  [9].  Metabolic  networks  belong  to  the  class  of  scale-free  networks  [14],  which 
means  that  their  degree  sequences  follow  a  power  law,  so  there  are  a  few  vertices  of  very  high  degree 
in  the  network.  These  vertices  are  called  hubs,  they  integrate  the  functionally  independent  modules 
of  the  metabolism  into  a  robust  network  [15].  Most  of  the  vertices  arc  connected  through  hubs  by  a 
relatively  short  path  [1],  [16]. 

In  figure  5  we  plotted  the  degree  sequences  of  metabolic  networks  of  C.  pneumoniae,  Escherichia 
coli  (E.coli)  and  Saccharomyces  cerevisiac  (S.  cerevisiae).  To  distinguish  functional  modules  based 
on  the  network  topology  alone,  several  hubs  are  usually  removed.  Cutting  vertices  of  highest  degrees 
increases  the  modularity  of  the  remaining  network  [6]  and  the  independent  functional  units  become  co¬ 
hesive  subsets  of  vertices  that  are  sparsely  interconnected  with  each  other.  In  a  preprocessed  network 
modules  can  be  distinguished  as  highly  connected  subsets  [15].  The  argument  behind  it  is  that  these 
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Figure  4:  Size  of  GSC  before  and  after  cutting  10  hubs 
The  data  was  fitted  with  the  lines  y  =  0.84x  —  6  for  the  original  netwwks  and  y  =  O.SSx  —  55  for  networks 

after  cutting  10  hubs. 

metabolites  have  secondary  role  in  synthesis  pathways  within  the  cell  [15].  Such  metabolites  are  for 
example  ATP,  ADP,  NADH,  NAD-f  and  have  primary  role  in  energy  supply  to  sustain  the  synthesis. 
Ill  [8],  [9]  current  metabolites  and  eofaetors  like  ATP,  ADP,  NADH,  NAD-h,  H2O  and  Pi  have  been 
removed  (small  molecules  such  as  H2O,  NH3,  O2,  CO2  and  phosphate  are  also  considered  as  current 
metabolites).  These  metabolites  function  as  carriers  for  transferring  electrons  and  certain  functional 
groups  (phosphate  group,  amino  group,  one  carbon  unit,  methyl  group  etc.).  The  elassifieation  of  cur¬ 
rent  metabolites  in  [8],  [9]  is  similar  to  the  elassifieation  of  internal  and  external  metabolites  in  [17], 
where  external  metabolites  are  removed.  These  are  metabolites  that  ean  be  regarded  as  externally 
buffered  with  respect  to  the  system.  Their  concentrations  ean  be  considered  constant  in  a  normally 
functioning  cell,  in  other  words,  they  put  virtually  no  constraints  on  network  dynamics.  Since  current 
metabolites  have  high  degrees,  they  are  represented  by  hubs  in  studies  where  abundant  metabolites 
have  not  been  removed  [6],  [14].  From  the  above  observation  it  follows  that  in  a  dynamically  mod¬ 
elled  metabolic  network  variables  representing  hubs  can  be  safely  removed  and  replaced  by  uncertain 
constants,  which  in  turn  means  that  the  corresponding  vertices  (and  edges  attached  to  them)  will  dis¬ 
appear  from  the  graph  of  the  metabolic  network.  We  call  a  degree  of  a  vertex  the  sum  of  its  in-degree 
and  out-degree.  The  plot  in  figure  4  shows  in  red  the  size  of  GSC  as  a  function  of  the  network  size  for 
the  42  networks  that  we  tested.  Blue  dots  show  the  size  of  GSC  after  cutting  the  first  10  hubs  from 
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a=84,  b=0.57,  c=-2.3 


Figure  5:  Degree  distributions  of  C.  pneumoniae,  E.coli  and  S.  cerevisiae 
The  degree  sequences  of  each  of  the  42  metaboHc  networks  that  we  have  tested  can  be  fitted  by  a  function  of 
type  t/  =  c  +  a/x^.  The  vertices  above  dotted  line  (hubs)  were  removed  from  the  network. 
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Figure  6:  HVD  graph  of  the  metabolic  network  of  E.Coli 


Figure  7:  HVD  graph  of  the  metabolic  network  of  S.  cerevisiae 

each  of  42  network  models.  The  data  was  fitted  with  linear  functions.  The  size  of  GSC  before  cutting 
the  hubs  behaves  approximately  as  y  =  0.84x  —  6,  and  the  size  of  GSC  after  cutting  10  hubs  behaves 
approximately  as  y  =  Q.85x  —  55. 

GSC  is  the  core  part  of  the  metabolic  network,  that  constitutes  the  dense  and  complicated  central 
unit  that  processes  a  large  number  of  metabolites  fanning  in  and  produces  from  them  many  metabolites 
that  arc  fanning  out.  One  can  observe  this  characteristic  on  HVD  graphs  of  metabolic  networks  of  C. 
pneumoniae,  E.coli  and  S.  cerevisiae  are  shown  in  figure  3,  figure  6  and  figure  7  respectively  (GSC  is 
shown  in  red).  The  metabolic  network  model  of  C.  pneumoniae  has  187  vertices  and  652  edges  see 
fig.  3,  GSC  has  136  vertices  and  551  edges.  The  metabolic  network  model  of  E.coli  has  766  vertices 
and  3988  edges  see  fig.  6,  GSC  has  630  vertices  and  3701  edges.  The  metabolic  network  model  of  S. 
cerevisiae  has  551  vertices  and  2806  edges  sec  fig.  6,  GSC  has  466  vertices  and  2631  edges.  The  type 
of  structure  observed  above  is  called  bow-tie  architecture,  it  is  the  key  design  principle  in  biological 
networks  that  allows  them  to  be  robust  yet  flexible  and  evolvable  [7]. 

In  figure  8  we  plotted  the  average  degrees  in  GSC  of  the  original  and  network  and  after  cutting 
10  hubs  from  all  networks,  except  C.  pneumoniae,  Helicobacter  pylori,  Methanococcus  jannaschii, 
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Figure  8:  Average  degrees  in  GSC  of  the  original  network  and  after  cutting  10  hubs 
Red:  average  degree  in  GSC  of  the  original  network 
Blue:  average  degree  in  GSC  after  cutting  about  10  hubs 
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Figure  9:  Ratio  of  number  of  edges  in  the  original  and  after  cutting  about  10  hubs 
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Streptocoeeus  pyogenes,  Treponema  pallidum  where  we  eut  9  hubs,  because  the  10th  and  11th  hubs 
in  these  networks  have  the  same  degree. 

Cutting  about  10  hubs  from  the  network  red  need  the  number  of  edges  almost  by  a  factor  of  2  for 
most  vertices,  and  even  more  for  smaller  networks,  see  figure  9.  This  suggests  that  we  eould  achieve  a 
better  reduction  of  number  of  edges  in  the  networks,  if  we  eut  a  larger  number  of  hubs  as  the  network 
size  grows.  In  particular,  the  number  of  edges  in  C.  pneumoniae  network  became  2  times  smaller  (323 
edges);  the  number  of  edges  in  E.  coli  network  became  1.7  times  smaller  (2386  edges);  the  number 
of  edges  in  S.  eerevisiae  network  became  1.7  times  smaller  (  1624  edges).  The  HVD  graph  of  C. 
pneumoniae  after  cutting  the  first  9  hubs  is  shown  in  figure  10.  The  GSC  of  this  preprocessed  network 
has  65  vertices  and  154  edges.  Which  means  that  the  GSC  graph  is  relatively  sparse,  with  average 
degree  4.7.  If  we  cut  the  first  20  hubs  in  E.  coli  network,  and  the  18  first  hubs  in  S.  eerevisiae  network, 
we  will  achieve  reduction  in  edge  number  similar  to  that  observed  in  C.  pneumoniae  after  cutting  9 
hubs.  The  number  of  edges  in  E.  coli  network  after  cutting  20  hubs  is  1804,  which  makes  a  reduction 
ratio  of  2.2.  The  GSC  of  E.  eoli  network  after  cutting  20  hubs  has  499  vertices  and  1424  edges,  which 
makes  the  average  degree  inside  GSC  5.7.  The  number  of  edges  in  S.eervisiae  network  after  cutting  18 
hubs  is  1259,  which  makes  a  reduction  ratio  of  2.2.  The  GSC  of  S.  eerevisiae  network  after  cutting  18 
hubs  has  381  vertices  and  1030  edges,  which  makes  the  average  degree  inside  GSC  5.4.  In  figure  11  is 
the  picture  of  a  HVD  graph  of  S.  eerevisiae  after  cutting  the  first  18  hubs.  In  figure  12  is  the  picture 
of  a  HVD  graph  of  E.coli  after  cutting  the  first  20  hubs.  After  cutting  several  hubs,  the  metabolic 
network  does  not  consist  of  one  eonneeted  unit  anymore,  but  its  largest  eoniieeted  component  still  has 
a  bow-tie  structure  with  a  giant  strong  component  in  the  center  of  the  network. 

HVD  decomposition  introduces  a  polarity  in  the  metabolic  network  graph.  If  we  eould  come  up 
with  a  decomposition  method  for  the  GSC  that  extends  the  polarity  introduced  by  HVD  inside  the 
GSC,  this  would  help  us  to  further  reduce  dimension  of  the  problem  for  variables  inside  GSC.  Next,  we 
propose  a  way  to  achieves  this.  We  will  illustrate  our  decomposition  method  on  GSC  of  C.  pneumoniae. 
First  we  locate  vertices  of  GSC  that  have  incoming  edges  from  outside  the  GSC.  We  call  these  vertices 
sources.  Next,  we  identify  vertices  that  have  edges  pointing  from  them  outside  the  GSC.  We  call  these 
vertices  sinks  (the  sinks  are  shown  in  blue  in  figure  13). 

We  put  sources  in  the  same  bottom  level,  then  we  perform  distance- labeling  of  the  vertices  in  GSC, 
starting  with  the  bottom  level.  In  each  step  we  construct  a  new  level:  it  will  include  all  vertices  that 
ean  be  reached  by  directed  edges  pointing  to  them  from  the  last  const rueted  level.  In  figure  13  is  shown 
the  decomposition  of  the  GSC  of  C.  pneumonia,  where  we  can  observe  a  clear  pathway  structure  in 
GSC.  The  decomposition  described  above  can  be  easily  combined  with  the  HVD  decomposition  of  the 
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Figure  10:  HVD  graph  of  the  C.  pneumoniae  network  after  cutting  the  first  9  hubs 


Figure  11:  HVD  graph  of  the  S.  cerevisiae  network  after  cutting  the  first  18  hubs 


13 


161 


B.5.  GRAPH  DECOMPOSITION  FOR  BIOLOGICAL  NETWORKS 


Figure  12:  HVD  graph  of  the  E.  coli  network  after  cutting  the  first  21  hubs 

entire  network.  Because  the  GSC  is  relatively  sparse,  there  are  few  edges  connecting  vertices  in  the 
same  level,  and  in  the  rest  of  the  GSC  we  uncovered  a  clear  pathway  structure,  see  figure  13. 

If  two  metabolites  are  connected  with  two  edges  pointing  in  opposite  direction,  one  of  these  edges 
must  indicate  the  direction  in  which  the  synthesis  goes  in  the  metabolism  under  normal  conditions 
(forward  edge),  and  the  function  of  the  other  edge  must  be  stabilizing  the  system  in  case  if  conditions 
change  (feedback  edge).  The  look  at  the  decomposition  of  GSC  from  dynamical  perspective  suggests 
that  most  likely  forward  edges  are  always  pointing  up  and  feedback  edges  are  pointing  down.  Of 
course,  further  tests  are  needed  to  confirm  our  hypothesis. 

4  Conclusion 

We  use  a  novel  approach,  HVD  decomposition,  to  reduce  dimension  of  problems  on  large-scale  biolog¬ 
ical  networks.  We  show  examples  of  HVD  decomposition  for  metabolic  networks  of  several  organisms, 
and  describe  how  to  use  it  to  reduce  the  dimension  of  the  problem  of  finding  the  trajectory  of  a  con¬ 
centration  of  a  given  metabolite.  The  bow-tie  structure  of  metabolic  networks  with  GSC  in  its  center 
is  reconfirmed.  We  show  that  cutting  current  metabolites  from  the  metabolic  network  does  not  affect 
its  dynamical  system  model.  Moreover,  cutting  as  few  as  10  hubs  from  the  network  decreases  the 
number  of  edges  in  the  network  about  by  a  factor  of  2  and  significantly  decreases  the  average  degree 
in  the  GSC.  This  allows  us  to  further  extend  the  HVD  decomposition  inside  the  new,  much  sparser, 
GSC.  We  state  as  hypothesis,  that  HVD  decomposition  and  its  extension  inside  GSC  reveals  forward 
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Figure  13:  Giant  strong  component  of  C.  Pneumoniae  network  after  cutting  9  hubs 
and  feedback  edges  of  the  network  based  only  on  structural  properties  of  the  metabolic  network. 
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We  propose  a  novel  graph  theoretic  decomposition  scheme  for  probing  the 
generic  structure  of  complex  biological  networks.  For  several  most  common 
cell  regulation  networks,  according  to  the  chosen  polarity  defined  by  the  input 
and  output  signals,  we  identified  the  ^^minimum  production  uniFX^P^)  which 
responds  quickly  and  robustly  to  external  signals  and  the  feedback  controllers 
which  adjust  the  output  of  the  MPU  to  desired  values  usually  at  a  larger  time 
scale.  Detailed  illustration  and  discussion  are  made  to  explain  the  network 
structures  and  how  they  are  tied  to  biological  functions.  The  proposed  scheme 
may  be  potentially  applied  to  parameter  evaluation  and  key  regulation  factor 
identification  in  a  variety  of  biological  systems. 

Cellular  behavior,  including  motility,  metabolism  and  reproduction  is  controlled  by  complex 
biochemical  reaction  networks,  many  of  which  have  been  identified  and  studied  in  detail  (I). 
These  networks  realize  their  regulatory  role  through  complex  molecular  interactions.  Con¬ 
temporary  high  through-put  experiments  produce  unprecedented  amount  of  data  that  serve  to 
pinpoint  the  players  and  their  interactions,  resulting  in  complex  chemical  reaction  graphs.  How 
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to  analyze  these  intricate  graphs  and  gain  insight  into  the  regulation  mechanism  employed  by 
cell  has  become  a  central  problem  of  the  molecular  biology. 

Much  progress  has  been  made  in  the  analysis  of  complex  networks,  both  deterministi¬ 
cally  (2, 3)  and  stochastically  (4-7).  These  studies  concentrate  on  investigation  of  dynamics  of 
given  networks  by  checking  their  stability,  parameter  dependence,  robustness,  input-output  re¬ 
lation.  However,  for  large-scale  networks  such  as  those  commonly  found  in  important  bilogical 
processes  (8,  9),  the  incurred  computational  load  often  severely  limits  detailed  analysis.  More 
critically,  with  continuous  experimental  efforts  of  revealing  details  of  a  network,  it  has  become 
increasingly  hard  to  identify  the  underlying  characteristic  structures  and  thus  gain  insight  into 
the  key  mechanism  that  shape  the  network  function.  Recently,  useful  concepts  distilled  from 
the  study  of  statistical  physics  such  as  the  small-world  and  the  scale-free  network  {10,11),  begin 
to  see  their  application  in  gene  regulation  network  and  lead  to  considerable  success.  However, 
this  type  of  statistical  analysis  mainly  aims  at  gross  features  of  networks  (72)  and  thus  ignores 
detailed  inhomogeneities  embedded  which  often  determine  the  functioning  of  a  network  in  an 
essential  way,  since  disparate  network  topologies  and  dynamics  fit  for  different  functional  re¬ 
quirements.  Other  methods  of  analysis  {13-15)  often  fail  to  highlight  interactions  between  the 
key  elements  of  a  system  or  properly  reflect  the  dynamics  associated  with  their  function. 

Normal  cell  life  involves  physical  or  chemical  activities  at  vast  range  of  spatial  and  temporal 
scales  and  one  central  task  of  systems  biology  is  to  identify  the  characteristic  structures  at 
all  scales  and  study  their  roles  in  relation  to  a  particular  cell  function  {16-21).  These  key 
structures  are  called  modules,  the  existence  of  which  contributes  almost  to  every  aspect  of  the 
cell  regulation:  robustness,  sensitivity,  adaptivity,  evolvability.  Their  detection  and  study  much 
simplifies  the  analysis  of  complex  networks  since  a  set  of  modules  could  be  a  lot  simpler  than 
a  collection  of  entangled  individual  agents  (22).  The  simplification  may  be  carried  out  further 
by  constructing  modules  of  modules.  Nevertheless,  the  determination  of  modular  structure 
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in  a  large  network  is  not  straightforward  since  one  molecular  species  may  be  involved  in  many 
different  pathways  with  very  distinct  external  connections.  Such  inter-correlation  is  easily  under 
appreciated  and  yet  can  have  profound  effects  on  the  organism. 

In  this  paper  we  propose  a  minimal  production  unit  (MPU)  -  feedback  controller  theory  of 
the  biochemical  networks  based  on  the  control  theoretic  point  of  view.  In  this  theory,  a  network 
is  decomposed  into  two  motifs:  one  is  the  pipeline  of  linear  production  unit  which  serves  to 
generate  the  output  in  a  quick  and  robust  way;  the  other  is  the  set  of  feedback  loops  which  act 
as  controllers  to  the  production.  These  two  motifs  are  decided  based  on  the  information  flow 
in  the  network.  Input  and  output  nodes  can  always  be  selected  in  a  cell  regulatory  network, 
which  defines  a  polarity  of  the  network.  The  information  is  received  at  the  input,  processed 
and  then  sent  to  the  output,  which  defines  an  overall  forward  direction.  The  units  that  carry 
on  the  information  along  the  forward  direction  belongs  to  the  production  motif,  while  the  re¬ 
maining  units  direct  the  information  in  the  opposite  direction  and  are  thus  treated  as  feedback 
controllers.  Every  module  in  the  network  belongs  to  either  of  the  two  motifs  and  can  be  further 
decomposed  in  a  similar  way  if  necessary.  In  this  way,  a  complex  network  can  be  analyzed  into 
a  hierarchy  of  modules  with  different  sizes  and  internal  dynamics.  From  biological  evolution 
point  of  view,  it  is  likely  that  this  nested  structure  stems  from  a  simple  core  and  is  later  wrapped 
with  complex  controller  in  evolution.  So,  our  theory  reveals  the  stable  generic  feature  of  a  bio¬ 
chemical  network,  which  can  be  used  to  explore  either  the  intricacies  of  a  single  structure  or 
interdependencies  of  a  series  of  systems.  In  the  following,  we  show  that  this  particular  structure 
are  accessible  via  graph  theoretic  analysis  and  revealing  special  production-controling  facilities 
in  cell  regulation  networks. 

The  dissection  of  large  networks  into  functional  modules  greatly  facilitates  their  analysis. 
The  functional  modules  can  be  studied  individually  with  well-designed  boundary  conditions. 
The  properties  of  the  whole  network  are  gathered  by  piecing  together  the  modules  in  an  ordered 
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way.  Henceforth,  our  strategy  of  analysis  is  characterized  by  a  decomposition  and  recombi¬ 
nation  procedure.  Below,  we  carry  out  the  decomposition  and  show  that  a  network  can  be 
algorithmically  dissected  into  disparate  functional  units.  The  horizontal-vertical  decomposition 
(HVD)  developed  in  a  recent  publication  (23)  decomposes  a  network  into  strongly  connected 
components  (SCC)  in  an  ordered  way,  displaying  the  underlying  pipeline  structure  at  the  largest 
scale.  To  look  into  each  SCC  of  interest,  we  design  a  new  scheme  which  first  searches  for  the 
embedded  cycles  and  then  determines  the  feedbacks  by  a  selection  procedure.  By  combined 
use  of  the  HVD  and  feedback  selection,  it  is  possible  to  identify  the  forward  edges  and  feed¬ 
backs  at  all  scales.  In  addition,  by  properly  cutting  certain  feedback  edges,  we  obtain  a  skeleton 
network  with  the  dominant  agents  and  key  interactions  identified,  as  well  as  their  ordering  and 
underlying  topological  structures,  which  is  called  a  minimal  production  unit  (MPU). 

In  the  following,  we  will  use  NFkB  as  an  example  to  explain  our  graph  theoretic  analysis 
procedure  and  display  the  production-controler  structure.  The  chemotaxis  network  of  E.  coli 
will  be  analyzed  to  further  show  this  universal  topological-dynamical  structure.  More  examples 
win  be  given  in  the  appendix. 
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Figure  1 :  The  diagram  of  a  model  of  the  NF-kB  signaling  module. 


The  NFkB  regulatory  module  concerns  the  switching  dynamics  of  the  nuclear  factor  NF/cB, 
which  regulates  various  genes  important  for  pathogen  or  cytokine  inflammation,  immune  re- 
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sponse»  cell  proliferation  and  survival  (24,25).  In  the  cytoplasm  of  a  resting  cell,  NFkB  usually 
binds  to  InBa  and  its  activity  is  suppressed.  Certain  external  signals  activate  the  switch  protein 
IKK  which  phosphorylates  I^Ba  such  that  NFkB  is  released  (26).  The  free  NFkB  then  translo¬ 
cates  into  the  nucleus  and  initiates  the  transcription  of  a  large  set  of  proteins,  including  protein 
lAcBa  and  protein  A20.  Protein  IkB^,  once  synthesized  in  the  cytoplasm,  enters  the  nucleus, 
binds  to  NFkB,  transports  it  out  to  the  cytoplasm  again  and  thus  terminates  the  transcription. 
Protein  A20  deactivates  IKK.  Therefore,  the  module  mainly  consists  of  two  forward  proteins 
IKK  and  NFkB  and  two  feedback  proteins  InEa  and  A20.  Also,  the  translocation  of  the  pro¬ 
teins  between  the  nucleus  and  the  cytoplasm  is  an  important  biological  process  that  achieves 
the  spatial  localization  of  different  protein  species. 

The  diagram  of  a  detailed  model  of  the  NFkB  regulatory  network  is  shown  in  Fig.  1(a) 
where  we  use  Xj’s  to  represent  the  concentration  of  various  proteins.  The  associated  chemical 
kinetic  model  is  given  and  explained  in  Appendix  A.  With  physiological  initial  conditions  (27), 
the  concentration  of  the  nuclear  NFkB  changes  as  a  damped  oscillation  as  shown  with  the  thin 
dotted  curve  in  Fig.  1(c).  At  the  beginning,  it  shoots  up  to  a  very  high  value  in  a  short  time  and 
then  relaxes  to  a  much  lower  steady  value  in  an  oscillatory  way. 

For  any  networked  system  described  by  certain  dynamical  equations,  it  is  easy  to  write  an 
interaction  graph  with  the  vertices  representing  the  reacting  agents  and  the  edges  directed  from 
each  agent  to  the  ones  influenced  by  it.  The  interaction  graph  for  the  NFkB  model  is  shown  in 
Figure  1(b). 

For  clarity,  we  omitted  the  self  loops  which  represent  self-interactions.  It  is  straightforward 
to  write  down  the  adjacency  matrix  for  the  interaction  graph,  which  marks  1  at  the  entries  cor¬ 
responding  to  connected  edges  and  zero  otherwise.  The  interaction  graph  and  the  adjacency 
matrix  neglect  details  of  the  interactions  and  only  maps  out  the  network  topology  which  holds 
almost  everywhere  in  the  phase  space  and  the  parameter  space,  except  for  a  set  of  measure 
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zero  (23).  This  great  robustness  confers  great  flexibility  to  characterize  vastly  different  dynam¬ 
ics  described  by  ODEs  or  mappings  or  even  stochastic  equations.  Certain  system  properties,  like 
the  uniqueness  of  the  stationary  point  sometimes  can  be  deduced  from  pure  topological  consid¬ 
eration  of  network  structures  (28,29).  So,  understanding  of  structure  of  interaction  graphs  helps 
unveil  the  essential  elements  in  a  complex  system  which  possibly  has  uncertainties  in  the  pa¬ 
rameter  values  or  is  influenced  by  a  noisy  environment.  Graph  theoretic  techniques  will  be 
developed  here  to  decompose  the  interaction  graph  and  disclose  its  generic  structure  used  to 
realize  its  biological  function. 

The  horizontal-vertical  decompx)sition  (HVD)  of  an  interaction  graph  has  been  contrived 
and  discussed  in  a  recent  paper  (23).  Vertically,  the  system  is  decomposed  into  a  linear  series  of 
modules,  where  the  module  above  is  influenced  by  the  module  below  but  not  vice  versa.  So,  the 
input  signal  propagates  unidirectionally  from  the  bottom  to  the  top.  Horizontally,  each  module 
is  decomposed  into  independent  groups  with  no  connection  between  them.  So,  each  group  has 
its  own  input  and  output  and  are  functioning  relatively  independently.  The  direct  application 
of  the  HVD  to  teh  interaction  graph  in  Fig.  1(b)  results  in  three  layers  with  the  top  and  bottom 
layer  consist  of  the  vertex  sets  {xi}  and  [x^  ^xis},  respectively.  The  rest  vertices  are  strongly 
connected  and  belong  to  the  middle  layer.  This  type  of  structure  with  dominant  intermediate 
processing  unit  exists  for  most  of  biological  networks  as  a  result  of  omnipresent  feedback  loops 
and  reversibility  of  many  biochemical  reactions. 

Further  analysis  is  needed  to  determine  the  forward  and  backward  edges  for  the  processing 
unit  in  the  middle  layer.  First,  its  polarity  is  easy  to  identify.  The  vertex  X15  is  the  output  signal 
that  is  of  interest  while  xy  receives  the  external  input.  Therefore,  in  the  middle  layer,  X2  is  the 
input  vertex  and  X7  is  the  output  one.  They  should  stand  at  the  beginning  and  the  end  of  the 
middle  module.  Next,  we  notice  that  in  a  strongly  connected  component  (SCC),  the  forward  and 
backward  edges  always  make  cycles  and  vice  versa  every  cycle  contains  at  least  one  forward 
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(a)  (b)  (c) 

Figure  2:  The  structure  decomposition  and  extraction  of  the  MPU  of  the  NF-kB  signaling 
regulatory  network,  (a)  the  structured  diagram  derived  from  the  graph  theoretic  analysis;  (b) 
with  feedbacks  removed;  (c)  with  irrelevant  vertices  removed. 

and  one  feedback  edge.  Since  cycles  are  obvious  topological  invariants  of  a  network  and  easy  to 
seek,  our  strategy  consists  of  two  steps:  first,  search  for  all  cycles  that  exist  in  the  graph;  second, 
determine  the  feedbacks  through  a  selection  procedure,  which  depends  on  the  information  flow 
direction  and  approximately  minimizes  the  number  of  feedbacks.  The  detailed  illustration  of 
our  technique  is  contained  in  Appendix  B.  Here  we  only  show  the  result  of  the  computation  in 
Fig.  2(a). 

From  Fig.  2(a),  we  see  that  there  are  4  feedback  loops: 

•  FBa  -  the  one  through  vertex  4:  IKKa  associates  with  free  InEa  and  catalyzes  its  decay. 

•  FB5  -  the  one  through  vertex  14:  lKBa„  captures  NFkB^  to  form  (iKBa-NFKB)^,  which 
then  moves  out  of  the  nucleus. 

•  FBc  -  the  one  through  vertex  12:  NFkB„  promotes  the  production  of  the  IkBo  mRNA 
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which  translocates  to  the  cytoplasm  and  initiates  a  burst  of  InBa  production. 

•  FBd  “  the  one  through  vertices  8  and  9:  NFkB^  promotes  the  production  of  the  A20 
mRNA  and  thus  initiates  the  production  of  A20,  which  catalyzes  the  decay  of  IKK^. 

This  identification  agrees  very  well  with  the  usual  recognition  of  feedback  loops  in  the  literature 
based  on  biological  reasoning.  In  Fig.2(a),  all  the  feedback  loops  FBa^b,c4  \QSiSl  one 

edge  attached  to  the  input  or  the  output  vertex,  so  at  least  4  cuts  are  needed  to  remove  these 
feedbacks.  Also,  there  are  7  extra  2-cycles  in  the  graph,  which  require  7  more  cuts.  So,  for  the 
NFkB  model,  at  least  11  cuts  are  required  to  remove  all  the  feedbacks.  With  our  algorithm,  we 
successfully  identified  a  minimal  cutting  set.  Those  big  cycles  are  uniquely  determined  while 
the  uncertainty  of  the  decomposition  comes  from  the  vertex  set  Sm  =  {3:5  ^Xq^Xiq^Xu  ,  X13} 
connected  by  those  2-cycles  exclusively. 

We  emphasize  that  we  acquired  the  network  structure  by  an  automatic  procedure  based 
on  the  graph  decomposition.  So,  the  method  developed  here  can  be  potentially  applicable  to 
real  complex  networks  in  cell  signal  transduction  or  gene  regulation.  It  is  also  understood  that 
because  of  the  existence  of  binary  or  dissociative  reactions,  the  rates  represented  by  some  edges 
are  correlated  since  they  symbolize  the  same  reactions.  In  the  above  consideration,  we  ignored 
this  correlation  and  carried  out  our  analysis  from  a  pure  graph  theoretic  point  of  view.  Further 
refinement  of  the  ordering  needs  to  incorporate  these  detailed  reaction  information,  as  shown 
below. 

After  the  structured  network  is  constructed  as  in  Fig.  2(a),  it  is  very  convenient  to  Extract 
the  minimal  production  unit.  In  the  case  of  signal  transduction  network,  the  minimal  production 
unit  is  the  minimal  subgraph  of  the  original  network  such  that  the  response  to  external  stimuli 
continues  to  be  produced  though  its  value  may  not  be  correct.  It  obviously  depends  on  the  initial 
state  of  the  system  and  on  the  specific  response  under  investigation. 
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In  the  NFkB  network,  there  exists  a  particularly  simple  procedure  to  extract  the  MPU  after 
we  identified  the  forward  and  backward  edges.  First,  the  feedbacks  and  the  associated  reac¬ 
tions  are  removed  as  we  now  only  consider  the  forward  production  part.  As  a  consequence, 
{x4 ,  xg  ,xi2 ,3:14}  are  removed,  which  results  in  Fig.  2(b)  where  the  correlation  among 
edges  have  been  considered.  Second,  all  the  outputs  except  the  one  we  are  interested  in  are 
removed.  That  is,  {x3,xu}  are  removed.  From  here  we  see  that  the  final  MPU  indeed  de¬ 
pends  on  what  signal  we  are  looking  at.  Finally,  remove  other  irrelevant  vertices.  It  is  Xiq  here 
since  its  initial  value  is  zero  and  does  not  receive  any  contribution  from  other  vertices  or  the 
environment.  The  arrow  from  xe  comes  from  a  binary  reaction  and  xio  can  not  be  produced 
there.  At  the  end  of  the  day,  we  produce  the  MPU  depicted  in  Fig.  2(c).  It  contains  the  vertex 
set  Sjn  =  {^"1 ,  X2  ,  X5 ,  xg ,  X7  ,  X13 ,  X14},  while  all  other  vertices  can  be  regarded  as  functioning 
controllers. 

To  check  if  what  we  got  in  Fig.  2(c)  is  indeed  the  MPU,  we  keep  only  the  variables  in 
the  vertex  set  Sm  and  their  interactions  in  the  evolution  equation.  Numerical  simulation  of  this 
reduced  set  of  equations  produced  an  output  curve  depicted  with  the  thick  solid  line  in  Fig.  1  (c), 
which  displays  a  fast  relaxation  to  a  large  value.  It  is  interesting  to  note  that  the  saturation  value 
and  the  relaxation  time  are  very  close  to  those  of  the  first  oscillation  peak  of  the  full  equationa. 
The  vertex  set  constitute  the  MPU  of  the  NFkB  gene  regulation  network,  which  generate 
a  quick  and  large  response  to  the  external  signal.  The  rest  vertieces  act  as  controllers  to  bring 
down  the  initial  pulse  to  a  desired  steady  value  in  a  much  longer  time  scale.  Both  the  short  and 
the  long  time  response  bear  important  biological  significances  (25). 

In  Ref.  (25),  it  is  pointed  out  that  a  two-component  system  with  a  negative  feedback  ex¬ 
hibits  the  basic  oscillating  or  saturation  behavior  depending  on  the  parameter  values.  Here,  the 
two  components  are  replaced  by  two  subnetworks  residing  in  cytoplasm  and  nucleus,  respec¬ 
tively.  The  forward  edges  and  feedbacks  are  realized  by  the  translocation  of  several  different 
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molecules.  As  extra  resources  are  needed  to  produce  these  regulatory  molecules,  the  question 
why  the  extra  feedback  loops  and  intermediate  steps  exist  is  naturally  raised.  External  signals 
could  have  directly  acted  on  NF/cB  to  control  its  activity.  Those  extra  features  in  network  struc¬ 
ture  and  interaction  dynamics  bring  about  extra  robustness  and  sensitivity  to  the  network  for 
fulfilling  its  basic  function  (iO). 

Lying  at  the  core  of  this  regulatory  network  is  the  association  and  disassociation  of  NF/cB 
with  I/^Ba.  When  binded,  they  stay  away  from  the  nucleus  and  the  NF^B-initiated  transcrip¬ 
tion  is  terminated;  when  they  separate,  NFkB  tends  to  enter  the  nucleus  and  starts  prompting 
transcription.  The  feedback  FB5  mainly  facilitates  the  step  of  clearing  NFkB  out  of  the  nucleus. 

The  feedbacks  FBc  and  FB^  are  two  negative  feedbacks.  FBc  is  to  restore  the  concentration 
of  IacBo  that  has  been  consumed  by  the  IKK^-catalyzed  decay.  FB^  is  to  deactivate  IKK^  by 
A20  to  bring  down  the  activation  level  of  the  whole  network.  Like  other  feedback  signaling  from 
the  output  {4, 31)  it  provides  adaptivity  and  sensitivity.  When  a  signal  such  as  TNF  just  arrives, 
IKK„  constantly  gets  activated  into  IKKq  while  the  deactivation  of  IKK^  is  minimized  since  its 
constitutive  decay  rate  is  small.  So,  the  concentration  of  IKK^  will  rapidly  increase  until  A20 
is  produced  by  the  feedback  loop  and  starts  the  catalyzed  decay  of  IKK^.  The  forward  reaction 
rate  is  thus  maximized  transiently  and  enables  cell  response  to  signals  with  short  duration  (25). 
The  negative  feedback  will  eventually  bring  down  the  IKK^  concentration  to  a  steady  level 
which  is  much  lower  than  the  transient  peak.  So,  the  network  has  a  very  sensitive  and  fast 
transient  response,  which  is  essential  for  certain  signaling  pathways  (25). 

The  oscillation  observed  in  this  process  is  a  signature  of  trading  stability  for  sensitivity  (27). 
It  confers  easy  excitability  to  the  network  while  brings  oscillations  at  the  same  time. 

The  reaction  rates  of  all  the  biochemical  processes  are  to  some  extent  influenced  by  the 
environment  variables  like  temperature,  pH  value,  concentration  of  certain  ions  (52).  To  work 
properly  under  different  conditions,  the  chemical  network  should  possess  structural  stability. 
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Here  the  double  feedbacks  FBc  and  FB^  offer  extra  structural  stability  against  parameter  uncer¬ 
tainty.  If  the  parameter  changes  incur  an  temporary  increase  of  the  concentration  of  NFkB„, 
then  both  FBc  and  FB^  will  act  to  bring  it  down.  Even  when  the  rate  of  FBc  or  FB^  changes, 
the  other  feedback  will  try  to  minimize  the  effect.  Therefore,  the  double  feedbacks  act  like  a 
double  safe  for  keeping  the  system  stable  under  parameter  fluctuations. 

The  above  procedure  of  searching  for  MPU  could  be  easily  generalized  to  more  complex 
networks.  The  critical  step  lies  in  our  capability  of  delecting  the  feedback  loops.  Once  the  feed¬ 
back  controllers  are  decided,  the  MPU  is  obtained  by  removing  all  the  feedbacks  and  then  all  the 
irrelevant  outputs  and  inputs.  The  exhibition  of  fast  forward  production  and  the  slow  feedback 
controller  is  also  quite  universal  as  demonstrated  by  the  next  example  of  E.  coli  chemoiaxis 
network,  as  well  as  those  in  Appendix  C. 


Figure  3:  (a)The  chemotaxis  model  of  E.  coli.  (b)  Feedback  and  forward  structure  through 
graph  decomposition,  (c)  The  response  of  CheYp  to  the  external  cue  of  the  full  network  (thick 
solid  line)  and  the  MPU  (thin  solid  line),  ligands  added  at  ^  =  500  and  removed  at  ^  =  1000. 


The  chemoiaxis  of  E.  coli  is  regulated  by  its  chemotaxis  pathway.  Chemoaiiraclanis  binds  to 
and  activates  the  iransmembrane  receptors,  which  stimulate  CheA  through  the  adaptor  CheW. 
Activated  CheA  phosphorylates  CheY,  which  binds  to  the  flagellar  motor  and  increases  the  fre- 
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quency  of  E.  coli  tumbling.  The  activation  of  the  receptor  complex  is  controled  by  its  methyla- 
tion  states.  Higher  methylation  states  indicate  higher  probability  to  be  activated.  In  the  model, 
CheR  binds  only  to  the  inactive  receptors  to  increase  methylation  and  phosphorylated  CheB 
only  to  the  active  receptors  to  decrease  methylation. 

The  chemotaxis  model  is  shown  in  Fig.  3(a).  Fig.  3(b)  displays  its  feedback  and  forward 
structure  upon  application  of  graph  decomposition.  The  first  level  constists  of  the  vertex  set 
{jTi  ,a:2  ,X3  ,X4  ^Xs}  which  are  different  methylation  states  of  the  receptor  complex.  External 
signals  propagate  down  through  xs  ,  xs  and  finally  reaches  the  flagellar  protein  xg.  There  is  one 
feedback  vertex  through  X7  (CheBp).  The  minimal  production  unit  (MPU)  is  obtained  after  all 
the  reactions  involving  X7  is  removed  and  is  contained  in  the  box  in  Fig.  ??(b). 

With  the  feedback  through  CheBp  (X7),  the  system  has  sensitive  detection  and  robust  adap¬ 
tivity  as  shown  with  thick  solid  line  in  Fig.  3(c).  Starting  with  zero  value  ,  the  CheYp  quickly 
reaches  the  saturation  level.  Ait  =  500.s,  an  external  stimulus  -  10//A/  concentration  ligand  is 
supplied,  which  induced  a  jump  of  CheYp  concentration  followed  by  an  exponential  decay  back 
to  the  saturation  value.  At  i  =  IOOO5,  the  ligand  is  removed  which  triggered  a  drop  of  CheYp 
concentration  but  regained  its  stable  value  exponentially  fast.  When  the  feedback  is  removed, 
the  MPU  retains  the  stable  value  after  a  quick  initial  rise  no  matter  how  the  concentration  of 
external  ligand  changes.  The  robustness  is  retained  but  the  sensitivity  is  lost.  So,  in  this  exam¬ 
ple  the  feedback  is  essential  for  system’s  transient  response  to  external  stimulus  and  guarantees 
the  sensitivity.  As  in  the  previous  example,  the  forward  production  reacts  quickly  accounting 
for  the  sensitivity  of  the  network  while  the  controller  effects  at  a  longer  time  scale  to  realize 
the  adaptivity.  This  has  been  observed  over  again  in  several  other  most  commonly  encountered 
biological  networks  presented  in  Appendix  C. 

We  devised  an  automatic  procedure  to  identify  the  key  functional  units  of  cell  regulation 
networks  by  applying  graph  theoretic  method  and  further  dynamical  systems  analysis.  We 
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identified  the  forward  production  pipeline  and  feedback  controller  in  the  NFkB  regulatory  net¬ 
work  with  the  HVD  and  the  feedback  loop  searching  algorithm.  They  form  disparate  functional 
modules  and  can  be  analyzed  further  with  the  same  program  if  necessary.  Based  on  the  gener- 
icity  and  topological  nature  of  the  anatomy  of  the  NFkB  network,  it  is  reasonable  to  believe  the 
existence  of  similar  structures  in  other  cell  regulatory  networks. 

The  detection  of  modular  structures  provides  additional  insight  how  a  regulatory  network 
works  and  thus  gives  clear  indication  of  key  protein  species  and  key  reactions  in  a  cascade, 
which  finds  vast  applications  in  the  drug  design  and  synthetic  biology  (ii).  The  disclosure  of 
the  dominating  skeleton  subnetwork  in  a  regulatory  pathway  also  avails  the  determination  of 
reaction  rales  of  in  vivo  biochemical  reaction  since  the  distracting  unimportant  reaction  com¬ 
ponents  have  been  removed  from  the  skeleton  structure  and  the  Jacobian  matrix  becomes  more 
balanced  (34). 

With  the  help  of  the  current  program,  the  analysis  of  large-scale  network  possibly  gains 
much  efficiency  by  building  the  hierarchical  connection  among  different  scales.  In  the  top-down 
direction,  the  network  may  be  broken  into  functional  modules  at  different  scales  by  the  above 
decomposition  technique.  From  bottom  up  after  the  property  of  each  module  is  conveniently 
explored,  a  hierarchy  of  modules  of  increasing  size  may  be  built  until  the  whole  network  is 
covered.  This  is  a  topological  generalization  of  multi-scale  analysis  (i5)  to  networked  systems. 
Future  application  of  the  decomposition  and  the  modular  analysis  technique  could  go  far  beyond 
biological  networks. 

This  reduction  is  closely  related  to  the  problem  of  determining  rales  of  in  vivo  chemical  re¬ 
actions  that  sensitively  depend  on  the  cell  conditions  and  are  hard  to  measure  directly.  They  are 
usually  estimated  based  on  the  response  curves  of  the  network,  with  known  interaction  graphs. 
A  generic  system  turns  out  to  be  very  sensitive  to  some  parameters  while  inert  to  others  (36). 
This  observation  is  pertinent  to  the  structural  and  functional  redundancy  commonly  found  in 
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biological  networks.  In  order  to  maintain  similar  biological  function  under  possibly  vastly  dif¬ 
ferent  environment,  a  typical  cell  regulatory  network  contains  much  more  extra  pathways  than 
is  needed  for  the  basic  activity  in  normal  conditions.  In  a  particular  experiment,  only  part  of  the 
network  properties  are  measured,  which  is  likely  to  be  carried  out  only  with  part  of  the  network 
so  the  rest  seems  redundant  and  the  corresponding  parameters  are  inert.  Our  current  approach 
could  be  used  to  determine  these  inert  parameters  and  thus  disclose  the  skeleton  of  the  network 
relevant  to  the  experimental  probation.  The  part  of  the  network  represented  by  the  inert  param¬ 
eters  can  be  viewed  as  adaptive  gears  which  take  effects  only  under  stress  conditions  and  are 
thus  negligible  in  normal  conditions.  In  the  following,  we  analyze  the  dynamical  system  Eq.  (1) 
to  excavate  the  inert  parameters  and  hence  pin  down  the  major  part  of  the  network. 

APPENDIX 
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A  Chemical  kinetic  equation  of  the  NF/cB  network 

Deterministically,  the  dynamics  of  the  NFkB  network  is  described  by  a  set  of  chemical  kinetics 


equations: 

Xi 

= 

^prod  ^degX\  k\UX\ 

^2 

= 

kiUX\  —  /C3X2  —  k2UX2X%  —  k^gX2  ~  ^2^2^10  "1"  ^1:^4  —  03X2X13  “h  t2X^ 

= 

k3X2  -f  A:20X2X8 

X4 

= 

02:^2X10  —  ^1X4 

= 

03X2X13  —  ^2^5 

xe 

= 

C^X\3  —  01X5X10  +  t2X3  —  i\X^ 

X7 

= 

i\kx:X^  — OiXyX]] 

Xs 

= 

C4X9  —  C5X8 

X9 

= 

C2  +  C1X7  —  C3X9 

^10 

= 

—  02^:2^10  “  OiXgXio  -h  C4aXi2  ””  Cs^Xio  ^la^lO  +  ^loXn 

xii 

= 

—  O1X7X11  -f-  “  Cia^'vXii 

x'n 

= 

C2a  +  ^XaXl  —  C3aX\2 

X\3 

= 

01X5X10  —  C5<2Xi3  —  O3X2X13  +  €,2aX\A 

x‘i4 

= 

O1X7X11  —  e2akvX\A 

Xi3 

= 

C2c  +  CicX7  —  C3<.Xi5  , 

where  the  notation  is  adopted  from  the  reference  (27)  and  marked  in  Fig.  1(a).  Protein  IKK 
has  three  different  forms,  the  neutral  form  IKK„  (xi),  the  activated  form  IKKa  (^2)  and  the 
deactivated  form  IKKj  (xa).  IKK„  is  passive  but  can  be  activated  into  IKK^  by  external  cues 
like  TNF  or  IL-l.  u  G  {0, 1}  is  a  switch  variable  which  is  equal  to  one  when  the  external 
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cue  is  present  but  equal  to  zero  otherwise.  IKKq  is  deactivated  by  A20  (xg)  into  IKKj  (X3) 
which  is  different  from  IKK„  and  cannot  be  activated.  IKKq  is  able  to  bind  with  I/cBa  (xio) 
to  form  the  protein  complex  I^Ba-IKKo  (X4)  or  with  the  complex  I^cBa-NF^B  (xia)  to  form 
a  tri-molecular  complex  IKKa-lAcBa-NPAcB  (xs).  Once  IKKa  binds  with  I^cBa,  in  either  the 
bimolecular  or  the  trimolecular  form,  it  phosphorylates  and  initiates  proteolysis  of  IkBq  so  that 
NFkB  (xe)  is  released  from  the  complex  and  restores  its  enzymatic  capability,  entering  the  nu¬ 
cleus  and  being  denoted  by  NFkB^  (xy).  When  NFkB^  binds  to  the  relevant  promoter  regions 
in  the  DNA,  a  great  variety  of  transcriptions  are  initiated.  The  transcripts  (xg)  for  A20  and  the 
transcripts  (X12)  for  lAcBa  move  to  cytoplasm  and  start  synthesis  of  the  corresponding  proteins. 
Other  interesting  transcripts  (xis)  for  certain  signaling  proteins  may  be  generated  as  well.  The 
newly  synthesized  protein  I/^Ba  will  enter  the  nucleus,  wherein  denoted  by  I^Ba^  (xn),  and 
bind  with  NF^B^  to  form  the  complex  I^Ba-NFAcB,^  (3:14),  leaving  the  nucleus  for  the  cyto¬ 
plasm.  Thus,  the  transcription  is  temporarily  terminated  by  the  association  with  protein  I^Ba. 
The  switch  protein  IKK^  itself  is  also  constantly  deactivated  by  A20.  In  the  mean  time,  almost 
all  the  proteins  and  transcripts  decay  spontaneously,  and  the  bindings  and  unbindings  all  have 
constitutive  reaction  rates  which  are  much  smaller  than  the  corresponding  enzymatic  reaction 
rates. 


B  Identifi  cation  of  forwad  and  feedback  edges 

In  this  appendix,  we  will  design  an  algorithm  to  identify  the  forward  and  feedback  edges  with 
given  polarity.  First,  a  cycle  search  program  is  discussed  which  produces  all  the  cycle  generators 
for  a  strongly  connected  component.  Then  a  selection  procedure  is  discussed  which  generates 
a  partial  order  of  the  vertices  and  enables  the  detection  of  feedbacks  in  a  straightforward  way. 
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B.l  Principle  of  minimum  feedbacks 

Very  often,  in  engineering  systems,  multi-step  processes  are  carried  out  in  a  well-ordered  se¬ 
quential  way  with  a  small  number  of  feedback  controllers  modulating  the  behavior  of  the  sys¬ 
tem.  The  cascade  structure  yields  both  robustness  and  evolvability  to  a  networked  system. 
It  also  has  the  advantage  of  maximizing  operation  efficiency  and  minimizing  energy  cost  by 
adjusting  magnitude  of  the  feedback  control.  As  an  analogue,  we  propose  that  a  minimum  feed¬ 
back  principle  is  also  utilized  by  the  cell:  the  feedback  edges  should  be  a  minimum  set  in  a 
biochemical  network  like  cell  regulation  networks. 

How  to  find  a  minimum  set  of  edges  is  an  NP-hard  problem  in  graph  theory  but  there  exist 
approximate  algorithms  which  could  do  the  Job  relatively  fast.  It  is  conceivable  that  the  solution 
might  not  be  unique. 

In  the  signal  transduction  network,  extra  constraints  are  imposed.  From  a  control  theory 
point  of  view,  the  signal  transduction  network  consists  of  two  major  components,  the  informa¬ 
tion  forwarding  part  and  the  feedback  controller.  The  forward  part  receives  external  signal  at 
one  end,  pass  and  process  it  along  different  paths,  produce  an  output  at  the  other  end.  So,  the  as¬ 
sociated  information  flow  defines  a  direction  on  the  network.  The  feedback  component  controls 
the  flow  by  sending  downstream  signals  back  to  upstream  nodes.  The  identification  of  these 
two  components  is  essential  for  understanding  the  function  of  individual  parts  of  a  network. 
So,  the  problem  of  searching  for  the  minimal  set  of  feedback  arcs  in  this  new  context  has  to  be 
consistent  with  the  constraints  brought  up  by  these  extra  features.  The  problem  could  be  put 
in  an  equivalent  way:  find  an  ordering  of  the  vertices  with  the  input  and  output  vertices  sitting 
at  opposite  ends,  such  that  the  number  of  feedback  edges  is  minimized.  To  achieve  this  goal, 
in  the  following,  the  HVD  and  the  cycle  searching  and  selection  techniques  are  discussed  and 
applied  to  the  graph  in  Fig.  1(b). 
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B.2  Cycle  search 

For  a  finite  graph,  all  cycles  can  be  obtained  by  algebraically  combining  a  set  of  independent 
cycle  generators.  For  an  SCC,  all  the  edges  are  included  in  the  generator  set  Cgen  since  there 
always  exists  at  least  one  cycle  passing  any  edge  and  thus  each  edge  is  contained  in  at  least  one 
generator.  In  the  following,  we  introduce  a  searching-collapsing  scheme  to  find  the  generators 
of  all  the  cycles  of  a  general  graph  ^  with  the  adjacency  matrix  A.  The  main  idea  is  to  identify 
shortest  cycles  and  then  simplify  the  graph  in  an  iterative  way: 

(1)  Record  all  the  self-loops  of  5^  which  are  encoded  by  the  nonzero  diagonal  elements  of  A. 
After  removing  the  corresponding  edges  from  we  obtain  a  new  graph  and  a  new  adjacency 
matrix  Ai. 

(2)  Search  and  record  a  shortest  m-cycle  Zi=[ai, am]  of  A\  where  aj’s  represent  vertices  of 
^  by  looking  for  the  nonzero  diagonal  elements  of  the  powers  of  A. 

(3)  The  induced  subgraph  by  the  vertex  set  {oi^ am}  has  an  adjacency  matrix  Bi  which 
is  a  submatrix  of  Ai.  Each  nonzero  element  {i,j)  of  Bi  represents  an  edge  from  the  vertex  aj  to 
the  vertex  a^  which  can  be  made  to  a  cycle  by  connecting  ai  back  to  aj  with  part  of  the  cycle  /i, 
eg.,  by  the  chain  of  edges  [a^,  a^+i, aj].  So  for  each  edge  in  but  not  in  /i,  we  can  identify 
and  record  a  cycle. 

(4)  Collapse  all  the  edges  and  vertices  in  the  subgraph  into  one  point  Pi,  and  we  obtain  the 
updated  graph  for  which  a  new  adjacency  matrix  A<i  is  written  down.  If  ^2  only  contains  Pi, 
the  iteration  is  terminated.  Otherwise,  we  go  back  to  step  (2)  and  repeat  the  procedure  with  the 
new  graph  ^2  and  the  new  adjacency  matrix  A2. 

It  is  easy  to  show  that  each  cycle  of  corresponds  to  a  unique  cycle  either  in  or  in 
Vice  versa,  each  cycle  /  in  ^2  can  be  identified  with  a  unique  cycle  in  if  the  cycle  I  runs 
through  Pi,  then  its  entering  vertex  and  leaving  vertex  in  can  be  connected  by  a  unique  path 
embedded  in  the  cycle  Zi  and  thus  a  unique  cycle  in  is  produced  by  concatenating  this  path  to 
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the  edges  contained  in  /;  if  the  cycle  I  stands  apart  from  Pi,  it  directly  corresponds  to  one  cycle 
in  So,  after  the  search  is  done,  finally,  we  can  trace  backward  all  the  cycles  contained  in  the 
original  graph  Here,  we  only  recorded  the  set  Cgen  of  cycle  generators.  As  we  pointed  out 
earlier,  however,  their  linear  combinations  are  able  to  produce  all  the  cycles  and  so  cover  the 
whole  transitive  part  of  the  graph.  The  generators  derived  from  the  above  algorithm  are  prime 
in  the  sense  that  any  proper  subset  of  a  generator  is  not  a  cycle.  Note  that  the  set  Cgen  is  not 
unique  since  the  selected  cycle  in  step  (2)  is  not  unique,  but  the  number  of  cycle  generators 
is  a  constant  which  depends  only  on  the  graph  itself.  The  important  point  here  is  that  all  the 
feedback  edges  appear  at  least  once  in  Cgen  as  argued  above. 

To  the  NF-kB  gene  regulatory  network,  we  apply  the  cycle-searching  technique  and  find 
that  the  total  number  of  cycle  generators  are  33  with  15  1-cycles  and  8  2-cycles.  Therefore,  10 
cycle  generators  has  length  longer  than  2. 

B.3  Selection  procedure 

In  order  to  determine  the  forward  and  feedback  edges  from  the  cycles  found  in  the  previous 
section,  we  utilize  the  polarity  of  the  network  that  has  the  input  and  the  output  points.  We  take 
the  middle  layer  of  the  NFkB  network  from  the  HVD  result  as  an  example.  Here,  the  input  point 
is  X2  as  it  receives  signals  from  xi  and  the  output  point  is  xj  as  it  sends  signals  to  X15.  The  goal 
is  to  find  all  the  forward  paths  that  go  from  X2  to  X7  and  all  the  feedback  loops.  The  problem  is 
clear  conceptually  but  not  rigorously  defined  mathematically  since  different  choices  may  lead 
to  different  forward  and  feedback  sets.  Here,  we  provide  one  choice  that  seems  reasonable. 
After  adding  a  direct  connection  (2 , 7)  from  Xj  to  X2,  we  apply  the  cycle  searching  technique 
developed  above  and  identify  a  set  of  cycle  generators. 

For  a  graph  with  tree  structure,  it  is  always  possible  to  find  an  ordering  of  the  vertices, 
such  that  feedback  edges  do  not  exist.  For  instance,  the  HVD  could  generate  such  an  ordering. 
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With  cycles  present,  at  least  one  feedback  exists  no  matter  how  the  vertices  are  ordered.  Here, 
we  implement  the  principle  of  minimal  feedbacks  to  extract  the  minimal  set  of  the  edges  the 
removal  of  which  leads  to  generation  of  tree  structures  in  the  network.  Therefore,  by  a  greedy 
algorithm  it  is  tempting  to  cut  those  edges  that  are  common  to  most  number  of  cycles.  It  is 
not  uncommon  that  by  removing  one  edge  quite  a  few  cycles  get  detroyed.  1 -cycles  (self¬ 
loops)  and  2-cycles  are  special  and  need  to  be  treated  differently.  1 -cycles  always  attach  to 
the  corresponding  vertices  and  are  not  regarded  as  feedback  loops,  the  2-cycles  correspond  to 
bidirectional  edges  most  likely  representing  the  forward  and  backward  reactions  since  many  of 
biochemical  reactions  are  reversible.  These  2-cycles  are  important  for  keeping  the  chemical 
balance  but  not  to  be  regarded  as  feedbacks  from  a  signal  transduction  point  of  view.  Each 
2-cycle  contributes  exactly  one  feedback  edge.  They  have  to  be  cut  one  by  one  irrespective  of 
the  ordering  of  the  vertices.  Therefore,  they  have  no  impact  on  the  vertex  ordering  regarding 
the  search  of  minimal  set  of  feedbacks.  Hence,  we  consider  only  cycles  of  length  larger  than  or 
equal  to  3  when  ordering  the  vertices.  In  a  graph  possessing  polarity,  for  the  input  vertex,  every 
out-edge  is  regarded  as  a  forward  edge  and  every  in-edge  a  feedback.  The  opposite  is  true  for 
the  output  vertex. 

With  the  long  cycles  having  been  determined  in  the  cycle  search  program,  we  first  look 
for  those  passing  the  edge  (2,7)  and  thus  obtain  a  set  of  forward  paths  that  go  from  X2  to 
Xj.  Then,  from  the  remaining  long  cycles,  we  search  for  cycle  segments  that  parallel  to  these 
already  obtained  forward  path  segments  so  that  alternative  paths  from  X2  to  xy  are  constructed. 
We  repeat  the  search  until  no  more  alternative  paths  can  be  generated  from  the  available  long 
cycles.  Now,  we  have  a  subgraph  ^  expanded  by  the  vertices  and  the  edges  contained  in  these 
forward  paths.  For  the  NFkB  regulation  network,  the  vertex  set  in  ^  has  been  computed  as 
Vj  =  {x2  ,  Xs ,  Xe ,  Xy  ,Xio ,  Xii  ,Xi3}  which  are  displayed  inside  the  rectangle  in  Fig.  2(a);  the 
complementary  vertex  set  consists  of  VJ,  =  {x4  ,X8  ,X9  ,xi2  ,xi4}.  They  do  not  belong  to  the 
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forward  pipeline  from  X2  to  X7,  so  they  must  be  included  in  the  feedback  motifs.  Next,  we 
partially  order  all  the  vertices  and  edges  according  to  the  determined  forward  path. 

First,  the  HVD  is  applied  to  We  get  4  layers  and  7  groups  with  one  vertex  in  each  group 
since  there  is  no  feedback  edge  in  So,  from  X2  to  Xy,  we  generated  a  partial  ordering  by 
the  layered  structure.  The  vertices  in  each  layer  are  not  ordered.  We  rearrange  the  order  of  the 
vertices  in  ^  according  to  the  partial  order  and  append  the  vertices  in  I4  to  the  end.  Using  this 
ordering  for  the  SCC,  a  new  adjacency  matrix  At  is  generated.  All  the  subdiagonal  entries  stand 
for  forward  edges  while  the  entries  above  the  main  diagonal  indicate  backward  edges.  We  can 
further  identify  all  the  2-cycle  edges  by  checking  the  diagonal-symmetric  partners  of  nonzero 
superdiagonal  entries.  If  the  subdiagonal  partner  is  also  nonzero,  then  together  they  stand  for  a 
2-cycle.  If  the  partner  is  zero,  then  we  have  a  long  feedback  loop,  the  collection  of  which  are 
clearly  exhibited  by  the  vertices  and  the  edges  outside  the  rectangle  in  Fig.  2(a). 

C  Examples  of  Several  other  cell  regulation  networks 

C.l  B.  Subtilis  chemotaxis  network 


Figure  4:  Network  representation  of  the  chemotaxis  model  of  B.  Subtilis. 
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Xi 
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Xg 

T,i 

Xii 

Motor  —  CheY 

X2 

Tio 

X7 

CheAp 

X12 

Ry 
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3^8 

CheBp 

^13 

RYp 

X4 

Toi 

Xg 

CheYp 

Xi4 

X5 

To2 

3^10 

CheVp 

Table  1 :  T,j  denotes  six  different  methylation  states  of  the  receptor  dimer  where  the  index  i  and 
j  mark  the  methelation  states  of  residue  630  and  637,  respectively.  CheABVY  denote  different 
kinases  in  chemotaxis  signaling.  RYp,  Ry-  denote  receptors  with  and  with  no  ChemYp  binding, 
while  R^  denotes  activated  recetors. 

The  chemotaxis  signaling  network  of  B.  subtilis  retains  many  features  of  that  of  E.  coli 
but  also  has  varied  considerably.  The  model  is  adapted  from  (37).  See  Fig.  4  for  the  inter¬ 
action  graph  and  the  notation  is  explained  in  Table  1.  Here,  in  a  similar  way,  the  receptor  (T) 
adopts  different  configurations  (active,  inactive,  weakly  active,  weakly  inactive)  according  to  its 
methylation  states  and  external  ligand  concentration.  However,  the  activation  and  deactivation 
of  the  whole  receptor  complex  (R)  also  depends  on  the  binding  of  CheYp.  The  active  receptor 
complex  can  activate  CheA  which  in  turn  activates  CheY.  The  phosphorylated  CheY  binds  to 
the  flagellar  protein  and  enhances  straight  runs  of  the  bacterium.  It  is  also  assumed  that  CheYp 
deactivates  CheA.  CheAp  also  activates  CheB  which  adjusts  the  methylation  state  of  the  recep¬ 
tor.  A  new  inhibitor  CheV  exists  in  B.  subtilis  network  which  disrupts  the  receptor  complex 
when  unphosphorylated  but  can  be  deactivated  by  CheAp  phosphorylation. 

Through  the  graph  decomposition  discussed  previously,  three  main  feedbacks  are  unam- 
buguously  detected  and  displayed  in  Fig.  5(a)  for  the  B.  Subtilis  chemotaxis  network.  One 
is  through  X8  (CheBp)  which  is  activated  by  X7  (CheAp)  and  changes  the  methylation  state  of 
the  receptor  (xi,...  ,6)*  The  feedback  through  Xio  (CheVp)  is  the  new  controller  of  the  receptor 
complex  while  the  feedback  through  X12  and  X13  relays  the  action  of  CheYp  (xg)  to  regulate  the 
activation  and  inactivation  of  the  receptor  complex.  The  first  six  variables  Xi,...,6  representing 
different  methylation  states  are  interconnected,  making  up  a  complete  subgraph  and  describing 
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(a)  (b) 

Figure  5:  (a)  The  feedback  and  forward  structure  through  graph  decomposition,  (b)  The  MPU 
of  the  B.  Subtilis  chemotaxis  network. 


their  interdependence.  All  of  them  affect  the  activation  of  the  whole  receptor  complex  (X14). 

We  checked  the  dynamics  by  using  the  kinetic  equations  and  parameter  values  in  {37).  Fig. 
6  shows  the  change  of  Xg  (CheYp)  in  the  time  interval  [0 , 1500],  starting  with  Xg  =  0,  IGjiM 
attactant  added  at  f  =  500  and  removed  at  f  =  1000.  When  the  full  network  is  put  to  work, 
the  adaption  is  clearly  exhibited  in  Fig.  6(a)  by  the  elevation  of  Xg  during  [500 , 1000]  in  the 
presence  of  the  chemo-attractant.  When  the  feedback  back  is  cut,  we  get  in  Fig.  6(b)  a  stable 
value  of  Xg,  which  does  not  change  with  the  attractant  concentration.  So  the  subgraph  shown  in 
Fig.  5(b)  is  an  MPU,  which  only  provides  the  basic  supply  of  CheYp,  devoid  of  capability  of 
adaptation.  Also  noticeable  is  the  fast  response  produced  by  the  MPU  and  the  relatively  slow 
adaptation  process  controled  by  the  feedbacks. 

Comparison  of  Fig.  ??  and  5  reveals  the  similarity  and  difference  between  the  chemotaxis 
pathways  of  E.  coli  and  B.  subtilis.  The  forward  MPU  parts  of  both  pathways  are  almost  iden¬ 
tical.  Nevertheless,  the  feedback  regulator  in  B.  subtilis  has  extra  loops  and  layers  which  may 
provide  more  subtle  control  {37). 
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t 


t 


Figure  6:  The  concentration  change  of  CheYp  (xg)  of  the  chemotaxis  network  of  E.  coli  (a)  with 
and  (b)  without  feedbacks.  External  attractant  of  is  added  at  ^  =  500  and  removed  at 

t  =  1000. 


C.2  Survival  and  apoptotic  pathways  initiated  by  TNF-a 

This  model  studies  the  survival  and  apoptotic  pathways  initiated  by  TNF-a  and  we  adopt  it 
from  which  play  decisive  roles  in  cell  fate  decision  in  response  to  inflammation  and  infec¬ 
tion.  After  the  external  cue  TNF-a  binds  to  its  receptor  TNFRl  (X2),  adaptor  proteins  TRADD, 
TRAF2  and  RIP-1  are  recuited  to  form  an  early  complex  ready  for  binding  and  activating  other 
functional  proteins.  There  are  two  different  downstream  pathways:  the  survival  pathway  medi¬ 
ated  by  NF-kB  and  the  apoptotic  pathway  mediated  by  caspase.  NF-kB  is  usually  sequestered 
by  IkB  and  is  released  when  IkB  degrades.  IKK  binds  to  the  early  complex  to  form  a  survival 
complex  and  is  activated  with  the  dissociation  of  this  complex.  The  activated  IKK  is  able  to 
induce  proteolysis  of  IkB.  The  released  NF-kB  translocate  to  the  nucleus,  binds  to  DNA  and 
leads  to  the  transcription  of  LAP  and  IkB.  c-IAP  inhibits  apoptosis  by  binding  to  caspase-3*  and 
thus  preventing  DNA  fragmentation.  The  notation  is  detailed  in  Table  C.2. 

Upon  application  of  the  graph  decomposition  routine,  we  unfold  the  successfully  unfold  the 
underlying  modular  structure  of  the  TNFa  network.  The  forward  production  unit  is  a  long  cas¬ 
cade  involving  many  different  species  and  reactions.  The  signal  TNFa  (xi)  is  processed  through 
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Figure  7:  Network  representation  of  the  Survival  and  apoptotic  pathways  initiated  by  TNF-a, 

the  network  until  DNA  fragmentation  is  induced  (X26)  as  shown  Fig.  8(a).  The  direct  HVD 
identifies  one  big  SCC  enclosed  in  the  two  boxes  in  Fig.  8(a).  Further  analysis  distinguishes 
the  forward  and  backward  edges.  The  whole  NFacB  pathway  is  now  treated  as  a  feedback  back 
module,  which  controls  the  level  of  the  c-IAP  {X27)  and  thus  Caspase-3*  (X25),  and  maintain 
the  option  for  survival.  It  is  intriguing  that  the  NFkB  module  is  produced  automatically  by  our 
decomposition  procedure  althougth  it  has  many  connections  to  the  rest  of  the  network.  The 
removal  of  the  NFkB  module  singles  out  the  MPU. 

Fig.  9  shows  the  level  of  DNA  fragment  (X26)  with  or  without  the  presence  of  the  NF«B 
control  module.  With  the  feedback  module,  the  fragmentation  of  DNA  is  low  (Fig.  9(a)), 
which  may  suggest  the  survival  of  the  cell;  without,  the  DNA  cleaverance  is  high  (Fig.  9(b)), 
which  could  indicate  an  apoptotic  fate  of  the  cell.  So,  indeed,  here  the  NFkB  modules  acts  as 
a  controller  of  the  apoptotic  pathway.  Our  decomposition  technique  accurately  captures  this 
information.  Again,  without  the  control  module,  the  MPU  produces  over  abundantly  the  output 
signal  in  a  relatively  fast  way.  The  long  feedforward  edge  from  xie  to  X27  may  accelerate  the 
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Table 

2:  TNFa  is  one  tumor  necrosis  factor  which  binds  to  the  receptor  TNFRl.  TRADD, 

TRAF2  and  RIPl  are  adaptor  proteins  which  may  form  complexes  with  TNFa.  IKK,  NFkB, 
IacB  belongs  to  the  NFkB  module  while  FADD,  caspaseS  and  caspase3  are  on  the  apoptotic 
pathway.  c-IAP  is  an  inhibitor  of  apoptosis  protein. 

control  in  this  case. 

C.3  Circadian  clock  in  Drosophila 

Circadian  clock  exists  in  many  different  organisms  ranging  from  bacteria  to  human.  The  reg¬ 
ulation  pathway  adopted  from  (i9)  and  displayed  in  Fig.  10  models  the  Drosophila  circadian 
clock  which  mainly  contains  two  interlocked  loops.  The  notations  are  explained  in  Table  3.  The 
TIM  and  PER  protein  in  the  first  loop  may  bind  to  each  other  in  the  cytosol  or  nucleus,  but  they 
enter  the  nucleus  separately.  They  down-regulate  their  own  expression  by  inhibiting  the  tran¬ 
scription  factor  CLK-CYC.  The  association  of  TIM  and  PER  in  the  cytoplasm  is  mediated  by 
FBM  and  the  dissociation  is  catalyzed  by  SM  which  is  generated  by  the  constitutive  entering  of 
PER  into  the  nucleus.  In  the  second  loop,  CLK-CYC  activates  both  VRI  and  PDP  expression. 
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(a) 


(b) 


Figure  8:  (a)  Feedback  and  forward  structure  through  graph  decomposition,  (b)  The  minima! 
production  unit  of  the  TNFa  network. 

VRI  represses  the  expresson  of  CLK  while  PDF  promotes. 

The  network  graph  after  the  decomposition  analysis  in  Fig.  1 1(a)  clearly  shows  5  feed¬ 
backs.  The  one  through  SM  (xn)  is  the  positive  feedback  that  accelerates  the  dissociation  of 
the  PER  TIM  complex.  The  other  four  are  the  important  regulators  of  the  concentration  of  PER» 
TIM,  VRI  and  PDF  through  DNA  expression  and  protein  translation.  The  feedbacks  through 
Xi2  and  Xi3  interact  with  each  other  and  control  the  production  of  CLK  (xig).  The  MPU  is  very 
easily  obtained  by  removing  the  feedback  modules  and  displayed  in  Fig.  1 1(b),  which  indicates 
how  the  (sunlight)  signal  is  picked  up  at  X4,  processed  via  PER-TIM,  CLK-CYC  interaction  and 
output  at  X21.  With  all  the  feedbacks,  the  Drosophila  network  is  able  to  generate  stable  oscil- 
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Figure  9:  The  evolution  of  the  DNA  fragment  (x26)  (a)  with  and  (b)  without  the  NFacB  feedback 
module. 
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Xg 
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VRI„ 

3^18 
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Table  3:  This  Drosophila  circadian  clock  model  consists  of  two  loops.  One  contains  PER 
and  TIM  and  the  other  contains  POP  and  VRI.  They  interact  through  CLK-CYC  controlled 
expression.  SM  and  FBM  are  two  proteins  assisting  in  the  first  loop. 


lations  with  period  being  24  hours.  Indeed,  employing  the  kinetic  model  in  {39)  and  starting 
with  a  somewhat  arbitrary  condition,  the  network  soon  reached  an  oscillatory  state  as  shown  in 
Fig.  12,.  Without  the  feedbacks,  all  the  state  variables  quickly  relaxed  to  a  steady  state.  The 
network  loses  its  function.  So,  these  feedbacks  are  essential  elements  for  the  generation  of  the 
circadian  cycles.  We  note  that  the  drift  to  stable  oscillation  in  Fig.  12(a)  takes  much  longer  than 
the  relaxation  in  Fig.  12(b),  which  indicates  that  the  MPU  works  at  a  much  smaller  time  scale 
while  the  feedback  controller  is  slowly  adjusting  the  motion  to  the  desired  one. 
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Figure  10:  Network  representation  of  the  Circadian  clock  in  Drosophila. 

C.4  The  EGF  receptor  signaling 

The  epidermal  growth  factor  receptor  (EGFR)  belongs  to  the  family  of  protein-tyrosine  kinase 
receptors,  which  regulate  cell  growth,  survival,  proliferation  and  differentiation  {40,41).  This  is 
a  simplified  version  of  the  EGFR  signal  transduction  network  adopted  from  {42).  The  interac¬ 
tion  graph  is  depicted  in  Fig.  13  and  notations  are  explained  in  Table  C.4.  Upon  binding  of  EGF 
(xi)  to  its  receptor  R  (X2),  the  occupied  receptors  EGFR  (X3)  will  form  dimers  EGFR2  (X4)  and 
get  phosphorylated  EGFR2p  (xs),  which  triggers  the  binding  of  the  cytoplasmic  proteins,  such 
as  Grb2  (x2o)»  Shc(x2i)  and  PLC7  (X22),  to  the  receptor.  PLC7  binds  to  the  receptor  to  form 
a  complex  EGFRp-PLC  (xe)  and  gets  phosphorylated  EGFRp-PLP  (xy).  The  activated  PLC7, 
i.e.  PLP  (xis)  then  dissociates  with  the  receptor  and  initiates  many  subsequent  reactions,  like 
binding  to  cytoskeletal  structures  PLC-I  (xig).  The  complex  EGFRp-G  (xg)  of  the  adaptor  pro- 
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Figure  11:  (a)  Feedback  and  forward  structure  through  graph  decomposition,  (b)  The  minimal 
production  unit  of  the  Drosophila  cirdadian  network. 


tein  Grb2  and  the  receptor  may  recruit  the  GDP-GTP  exchange  factor  SOS  (X22)  and  put  it  in 
the  vicinity  of  Ras.  The  binding  of  Grbs  to  the  receptor  could  also  be  mediated  by  She,  which 
makes  a  complex  EGFRp-Shc  (xio)  first  and  is  consequently  phosphorylated  into  EGFRp-ShP 
(xii).  The  binary  complex  is  able  to  bind  Grb2  to  form  a  ternary  complex  EGFRp-ShP-G  {x\2) 
and  then  binds  to  SOS  to  produce  a  quadruple  EGFRp-ShP-G-S  (x^),  which  approaches  and 
affects  the  Ras  activity  like  as  well  as  in  the  complex  EGFRp-G-S  (xg).  All  the  complexes 
dissociate  at  some  rates. 

In  this  example,  almost  all  reactions  are  reversible  so  the  whole  network  is  strongly  con¬ 
nected  and  hence  usual  simple  decompostion  schemes  are  hard  to  work.  Our  scheme,  however, 
is  still  able  to  generate  the  biologically  meaningful  topology  if  the  input  and  ouput  vertices  are 
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Figure  12:  The  evolution  (a)of  the  cytoplamic  CLK-CYC-Pc  (X21)  with  all  the  feedbacks  present 
and  (b)  of  ^23  without  the  feedback  module. 


Xi 

EGF 

X7 

EGFRp-PLP 

^13 

EGFRp-ShP  -  G  -  S 

X2 

R 

^8 

EGF^-G 

Xi4 

G-S 

^3 

EGF-R 

Xg 

EGFRp-G  -  S 

^15 

ShP 

X4 

EGF  -  R2 

Xio 

EGFl^-Shc 

^16 

ShP  -  G 

EGF  -  R2p 

Xu 

EGFRp-ShP 

Xi7 

PLC 

xe 

EGFRp-PLC 

X12 

EGFRp-ShP  -  G 

PLP 

PLC-I 

G 

She 


S 

She  -  G  -  S 


Table  4:  R  denotes  the  EGF  binding  receptor.  PLC  and  PLP  denotes  the  enzyme  PLC^  and  its 
phosphorylated  form,  respectively.  G  and  S  stand  for  Grb  and  SOS.  ShP  is  the  phosphorylated 
form  of  She. 

appropriately  selected,  see  the  decomposed  graph  in  Fig.  14(a).  The  binding,  dimerization  and 
phosphorylation  of  the  EGFR  leads  to  the  production  of  EGF-R2p  (X5)  which  becomes  the  hub 
of  all  the  signaling  processes.  The  reactions  involving  PLC  (xu)  are  automatically  selected  to 
make  one  separate  module  in  the  small  box  on  the  right  of  Fig.  14(a)  since  they  are  not  direcly 
related  to  the  production  of  EGFR2p-G-S  (xg).  Displayed  in  the  graph  are  mainly  three  routes 
to  the  target  output  xg.  The  longest  one  is  through  X21  — >  xio  — >  xn  — >  X12  — ^  X13,  another 
is  through  xg  and  the  shortest  one  is  a  direct  connection  from  X5  to  Xg.  But  the  shortest  one  is 
related  to  the  reverse  reaction  of  the  dissociation  of  the  EGFR2p-G-S  complex  and  should  not 
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Figure  13:  Network  representation  of  the  epidermal  growth  factor  receptor  model. 

be  treated  as  a  signaling  process.  It  is  removed  in  the  MPU  graph  shown  in  Fig.  14(b).  So, 
there  are  two  major  pathways  as  explained  before:  the  short  one  passing  Xg  is  through  direct 
binding  of  Grb  and  the  other  long  one  is  mediated  by  the  binding  and  activation  of  She.  The 
MPU  shown  only  contains  the  short  course. 

The  time  evolution  of  the  output  plotted  in  Fig.  15  shows  that  with  the  feedback  control 
(Fig.  15(a)),  the  signaling  arrived  at  the  desired  value  Xg  =  0.13  in  a  short  time  and  stay  there 
indefinitely  long.  Working  only  with  MPU  (Fig.  15(b)),  however,  the  signal  quickly  rises  to 
a  large  steady  value  xg  =  4.5  and  might  change  the  behavior  of  the  downstream  reaction  as  a 
consequence.  As  before,  the  fast  production  unit  responds  in  a  rapid  and  uncontroled  manner 
while  the  extra  controllers  may  pin  down  the  output  to  the  desired  value  at  a  large  time  scale. 
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Figure  14:  (a)  Feedback  and  forward  structure  through  graph  decomposition,  (b)  The  minimal 
production  unit  of  the  EGFR  network. 

C.5  G-protein  coupled  receptor  model 

Here,  the  model  is  taken  from  (43),  for  the  G  protein-coupled  receptor  (GPCR)  signaling  mod¬ 
ules  in  macrophage  immune  cells.  The  GPCR  system  responds  to  multiple  external  cues,  such 
as:  light,  hormones,  odorants,  neurotransmitters,  amino  acids,  which  is  an  important  drug  target 
and  is  one  of  the  most  common  signaling  channels  in  a  cell.  Here,  a  simplified  model  is  used  to 
study  the  effects  of  two  signaling  molecules  (C5a  and  UDP)  on  the  second  messenger  calcium 
Ca2+. 

The  interaction  graph  of  the  mod^  network  is  displayed  in  Fig.  16  and  the  notation  is  ex- 
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Xi 

C5aR 

^21 

X2 

C5aC 

X22 

^3 

GRKp  •  G^7 

^23 

X4 

GRKp  •  G^7  •  C5aC 

X24 

^5 

C5aCp 

3:25 

X6 

P2YR 

^26 

X7 

UDPC 

X27 

^8 

G^7  •  GftiGDP 

^28 

Xg 

G^7 

^29 

^10 

GaiGTP 

^30 

Xii 

GrtiGDP 

^31 

X12 

G^7  •  Ga,GDP 

^32 

^13 

Ga,GTP 

^33 

Xi4 

Ga,GDP 

^34 

^15 

RGSa 

^35 

^16 

RGSa  •  Ga.GTP 

3*  36 

Xi7 

RGS„  •  Ga,GTP 

^37 

^18 

PLC;04 

^38 

^19 

Ca2+ 

3^39 

^20 

PLC/34  •  Ca^+ 

X40 

PLC/34  •  •  Ga,GTP 

PIP2 

PLC^4  •  Ca2+  •  Ga,GTP  •  PIP2 
IPS 
DAG 
PLC^S 
PLC^S  •  Ca^+ 

PLC/33  •  Ca^+  •  Ga,GTP 
PLC^3  •  Ca^^  •  Ga,GTP  •  PIP2 
PLC^S  •  Ca2+  •  G^7 
PLC/93  •  Ca2+  .  G/97  •  PIP2 
PKC  •  DAG  •  Ca2+ 

PKC  •  DAG  •  Ca2+  •  PLC/34  •  Ca^+ 
PLC/34  •  Cal+ 

PKC  •  DAG  •  Ca2+  •  PLC/33  •  Ca^+ 
PLC^3  •  cal+ 

IP3R 
IP3R  •  IPS 
IP3R  •  IPS  •  Ca^+ 

IP3R  •  Ca^+ 


X41 

Buf 

^42 

Ca^+  ■  Buf 

^43 

Gagfi 

X44 

PKC 

X45 

PKC • DAG 

X46 

PKC  •  Ca^+ 

X47 

GRKp 

^48 

GRK 

X49 

PKC  •  DAG  •  Ca^+ 

^50 

DAGd 

^51 

IPSA'a 

^52 

IP4 

^53 

IPS 

Table  5:  The  notations  for  the  G-protein  coupled  receptor  model.  Buf  represents  other  calcium 
buffers  in  the  cell. 
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Figure  15:  The  evolution  (a)of  the  EGFRp-G-S  (xg)  with  all  the  feedbacks  present  and  (b)  with 
MPU. 

plained  in  Table  5.  The  main  biochemical  process  could  be  explained  as  following.  Upon  C5a 
and  UDP  binding  to  specific  cell  surface  receptors,  the  G-protein  heterotrimer  dissociates  to 
free  Ga-GTP  and  both  being  able  to  bind  isoforms  of  PLC/3  and  catalyze  the  synthesis  of 
IP3  and  DAG  from  PIP2.  IP3  binds  to  certain  ion  channels  on  the  membrane  of  endoplasmic 
reticulum  to  induce  the  release  of  Ca^"*"  into  the  cytosol.  DAG  and  Ca^"*"  bind  to  and  activate 
PKC  which  then  phosphorates  and  inactivate  PLC/3.  GRK  is  localized  at  the  membrane  by  G/37 
once  it  is  phosphorylated  by  PKC.  GRK  phosphorylates  and  inactivates  C5a  receptor  (C5aR). 
There  are  also  Ca^'*’  flow  from  other  buffers  in  the  cell  or  from  extracelluar  environment. 

Upon  application  of  our  decomposion  technique  to  the  above  network,  a  structured  graph  is 
obtained  and  displayed  in  Fig.  17.  The  network  consists  of  four  parts: 


•  The  source  vertex  set  {xe ,  xy  ,  X51 }. 

•  The  sink  vertex  set  {xso}. 

•  The  feedback  vertex  set  {xs  ,  X34 ,  X41 ,  X42  ,  X48  ,  X49  ,  X52  ,  X53}. 

•  The  major  strongly  connected  processing  unit,  the  rest  of  the  vertices 
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Figure  16:  Network  representation  of  the  G>protein  coupled  receptor  model. 

In  the  major  processing  part,  the  cycle  search  and  selection  program  decomposed  it  into  hor¬ 
izontal  layers,  being  put  in  an  order  which  relays  the  signals  input  from  vertex  Xi,  the  cell 
surface  receptor  and  terminates  at  vertex  xig,  the  calcium  ion.  According  to  the  order  given  in 
the  figure,  we  can  see  that  there  are  many  feedbacks.  It  happens  that  the  feedback  vertex  set 
mark  the  major  ones.  The  feedback  through  X52  ,^53  marks  the  feedback  of  phosphorylation 
of  IP3  back  to  PIP2;  the  one  through  X4s  ,  ^49  marks  the  feedback  of  PKC  on  GRK2;  the  one 
through  X34  marks  the  action  of  calcium  on  isforms  of  PLC/3;  the  one  through  0:41 ,  X42  marks 
the  exchange  of  calcium  with  other  calcium  buffers.  Of  course,  there  are  many  other  important 
feedbacks.  They  are  visible  as  feedbacks  in  the  boxed  forward  processing  unit.  The  MPU  is 
shown  in  Fig.  17(b)  where  we  have  removed  the  feedback  box  and  some  of  the  feedbacks  in  the 
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forward  processing  unit  which  are  unidirectional.  Shown  in  Fig.  18(a),  is  the  change  of  Ca^"*" 
over  time  when  a  concentration  250nAf  of  C5a  is  added  at  ^  =  0.  A  fast  initial  growth  of  Xig 
and  a  slow  relaxation  to  a  steady  value  is  obserbed.  With  only  the  MPU,  X\g  reaches  a  very 
high  value  in  a  short  time  and  stay  there  without  being  brought  down  to  the  equilibrium  value 
of  the  full  network.  So,  those  feedbacks  act  as  a  controller  to  set  up  the  output  of  the  system 
to  a  controled  value.  The  MPU  responds  to  external  signals  with  fast  production  as  in  other 
examples. 

C.6  A  phage  decision  circuit 

When  the  bacterium  Esherichia  Coli  is  infected  by  phage  A,  there  are  two  possible  subsequent 
pathways:  lysogeny  and  lysis  depending  on  the  concentration  of  particular  proteins.  In  the  lyso¬ 
genic  pathway,  the  DNA  of  the  A-phage  will  be  integrated  to  the  host  DNA.  In  the  lytic  pathway, 
the  host  DNA  will  be  excised  to  provide  material  for  A-phage  duplication.  The  selection  of  the 
pathway  is  made  by  the  A  lysis-lysogeny  decision  circuit  where  the  molecular  fluctuations  play 
significant  roles  (44). 

Here,  we  present  a  circuit  model  due  to  H.  H.  McAdams  and  L.  Shapiro  (45). 

Upon  infection  of  E.  coli,  phage  A  is  faced  with  selection  of  two  different  fates:  lysis  where 
more  copies  of  phage  A  are  assembled  and  released  after  the  host  bacterium  is  lysed  or  lysoge- 
nesis  where  the  DNA  of  the  phage  is  integrated  into  the  host’s  DNA.  This  fate  decision  is  made 
by  a  well-characterized  competitive  regulatory  mechanism  of  a  bistable  gene  switch.  The  core 
of  the  switch  is  the  Pr  and  Prm  operators  which  share  three  operator  sites  (ORl ,  OR2,  OR3). 
Translation  of  Pr  transcript  produces  Cl  while  encodes  Cro.  The  two  factors  Cl  and  Cro 
bind  to  the  operators  sequentially  in  the  opposite  order.  When  Cl  binds  to  ORl ,  is  repressed 
following  which  Prj^j  is  activated  by  Cl  at  OR2  and  repressed  by  Cl  at  OR3.  Prm  is  repressed 
when  Cro  binds  to  OR3  and  P/?  is  repressed  by  Cro  at  OR2  or  ORl.  Initially  Prm  is  off.  ButCII 
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^26 
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^36 
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Xj 
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Xl7 
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X27 

P 

X37 
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Xs, 

ciii 

X\S 

n 

X28 

q 

X3S 
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X9 

CIII 

Xl9 

N 

X29 

Q 

X39 

S 

XlO 
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X20 
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X30 

anti  —  Q 

Table  6:  In  the  table,  Pv4Q»P/?'  are  promoters;  tRl,  tR2,  tLI,  tL2,  tR’  are 

terminator  on  the  DNA  strand;  Cl,  CII,  CIII,  N,  O,  P,  Q,  Cro,  Xis,  Int,  Anti-Q  are  regulatory 
proteins;  Anti-Q,  Ant-Cro  are  gene  transcripts.  The  corresponding  gene  is  marked  by  lower 
case  letters. 

initiated  from  Pr  and  CIII,  N  initiated  from  Pi  lead  to  the  stimulation  of  Pre,  which  induces 
rapid  production  of  Cl  and  suppresses  the  production  of  Cro  by  the  anti-Cro  transcript.  Both 
Cro  and  Cl  supress  Pi.  Protein  N  is  able  to  anti-terminate  the  terminators  tRl,  tR2,  tLI,  tL2, 
such  that  CII  and  CIII  could  be  produced  rapidly  and  the  production  of  proteins  Q,  Xis,  Int 
is  able  to  start.  Regulator  Q  can  anti-terminate  tR’,  which  paves  the  way  for  the  P/?/  initiated 
transcription  to  continue  through  S  and  activate  the  lysis  coding  genes.  However,  Q  may  be 
repressed  by  Anti-Q  initiated  from  P^q  which  is  activated  by  CII.  The  concentrations  of  Xis 
and  Int  will  determine  lysis-oriented  excision  or  lysogenesis-oriented  integration,  respectively. 
The  CII  activated  promoter  P/  favors  the  production  of  Int. 

In  all,  roughly  speaking,  Cl,  CII  and  CIII  production  leads  to  lysogenesis  while  Cro  pro¬ 
duction  leads  to  lysis.  Both  the  bacteria  state  such  as  the  multiplicity  of  infection  (MOI)  and 
external  cues  such  as  ultraviolet  light  play  important  roles  in  the  fate  decision  process.  Many 
components  of  the  network  are  effective  only  transiently.  Once  the  decision  has  been  made, 
they  are  suppressed.  In  the  model  shown  in  Fig.  19,  the  operators  ORl,  OR2,  OR3  are  not 
represented  explicitly. 
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This  network  is  quite  sparse.  Xi  represents  the  external  signal  source  and  sits  on  the  top 
of  the  decomposed  structure  in  Fig,  20(a).  Five  elements  {X25  ,  X27 ,  X36  ,  X38  ,  X39}  stand  at  the 
bottom,  in  which  {X25  ,  X27}  are  intermidate  products  that  are  not  active  in  the  model  while  the 
rest  are  important  for  the  A  lysis-lysogen  fate  decision.  The  main  processing  part  is  contained  in 
the  left  long  bar  where  the  signal  enters  through  cii  (xs)  and  finally  switches  on  (off)  N  (xig)  and 
tRl  (x22)-  All  the  subsequent  steps  depend  on  these  two.  The  feedback  vertices  are  contained  on 
the  small  vertical  box  to  the  right  of  the  main  processing  unit.  The  Cro  (xu)  autoregulation  and 
the  cm  factor  (xg)  are  treated  as  feedbacks,  which  seem  plausible.  Certainly,  different  choice  of 
the  input  or  the  output  vertices  may  result  in  different  sequence  and  partitions.  Also  noticeable 
is  the  small  loop  in  the  main  unit  consisting  of  {xi4  ,X2  ,X3  ,X4}  with  one  feedback  from  X4 
to  X14.  This  loop  is  just  the  autoregulation  module  of  Cl.  After  removing  most  feedbacks,  the 
MPU  is  obtained  and  displayed  in  Fig.  20(b)  where  the  signal  relaying  process  is  very  clearly 
exhibited.  As  before,  the  MPU  is  very  likely  losing  its  biological  function  without  the  feedback 
controllers. 
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(a)  (b) 

Figure  17:  (a)  Feedback  and  forward  structure  through  graph  decomposition,  (b)  The  minimal 
production  unit  of  the  G-protein  coupled  receptor  network. 


43 


209 


B.6.  UNFOLDING  CELL  REGULATION  NETWORK  ANATOMY  THROUGH  GRAPH 
DECOMPOSITION 


Figure  18:  The  evolution  of  the  (xig)  with  all  the  feedbacks  present  (a)and  only  with  MPU 
(b). 
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Figure  19;  Network  representation  of  the  A  phage  decision  circuit. 


45 


211 


B.6.  UNFOLDING  CELL  REGULATION  NETWORK  ANATOMY  THROUGH  GRAPH 
DECOMPOSITION 


(a)  (b) 


Figure  20:  (a)  Feedback  and  forward  structure  through  graph  decomposition,  (b)  The  minimal 
production  unit  of  the  A  lysis-lysogen  decision  network. 
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Constrained  Dynamics  Lifting 


Gear,  Given  and  Kevrekidis 
August  3,  2008 


Abstract 

In  this  paper  we  present  a  novel  numerical  scheme  that  aims  on 
integrating  a  dynamical  system  which  exhibits  time  scale  separation. 
The  novelty  of  the  new  algorithm  is  that  it  assumes  only  the  separation 
of  scales,  without  explicitly  knowing  what  variables  are  the  slow  ones 
and  what  variables  are  the  fast  ones. 


1  Introduction 

Many  problems  in  science  can  be  described  by  a  dynamical  system  that 
exhibits  time  scale  separation.  The  case  where  the  variables  can  be  cate¬ 
gorized  as  slow  and  fast  variables  has  been  successfully  treated  in  the  past 
years  mainly  due  to  tools  provided  by  Differential  Algebraic  Equations  and 
the  Averaging  Principle.  Various  kinds  of  multiscale  algorithms  have  been 
introduced  to  effectively  integrate  systems  with  scale  separation.  Follow¬ 
ing  the  introduction  of  a  novel  auxiliary  process  by  Freidlin  and  Wentzell 
[1],  different  integrators  were  suggested  in  [2],  [3], [4]  which  are  based  on  a 
numerical  discretization  of  the  auxiliary  process.  The  use  of  the  averag¬ 
ing  principle  in  this  context  crucially  depends  on  many  layers  of  a  priori 
knowledge  about  the  evolution  equations,  for  example,  the  decoupling  of 
the  variables  into  slow  variables  and  fast  variables,  the  full  knowledge  of  the 
different  summands  on  the  right  hand  side  of  the  evolution  equations  for  the 
slow  variables  which  in  turn  implies  the  a  priori  knowledge  of  the  nature  of 
the  slow  variable  (deterministic,  stochastic,..)  and  last,  the  assumption  that 
the  observables  are  the  slow  and  fast  variables  of  the  system. 

Scientists  and  engineers  are  faced  with  research  problems  that  often  have 
many  complex  internal  feedback  processes  that  defy  simple  analysis,  or  that 
must  be  studied  at  scales  that  are  much  different  than  processes  occurring 
in  nature.  In  such,  any  a  priory  assumption  on  the  nature  of  the  coarse 
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description  limits  the  feasibility  of  the  results.  These  challenges  often  need 
massive  datasets  for  simulation,  have  heterogeneous  data  sources  that  must 
be  linked,  or  generate  massive,  high-dimensional  datasets  from  experiment 
or  observation,  and  will  soon  be  beyond  today's  capabilities. 

Computational  experimentation  allows  insight  into  complex  systems  by 
enabling  the  creation  of  a  virtual  description  (algorithmic  or  computational) 
that  can  interact  with  elements  from  the  real  world.  Simulation  and  other 
dynamic  modeling  techniques  allow  us  to  experiment  with  complex  systems 
in  ways  that  would  be  unimaginable  in  the  real  world,  and  to  constrain  our 
understanding  of  the  system  characteristics  or  underlying  physical  phenom¬ 
ena.  Furthermore,  it  allows  us  to  guide  real  world  operations  and  exper¬ 
imentation  in  cases  that  have  potential  for  unforeseen  or  extreme  events. 
Research  in  this  area  will  provide  needed  new  modeling  techniques  ranging 
from  mathematical  formulations  to  multi-scale  simulation  techniques. 

A  novel  approach  which  is  based  on  computational  experimentation  is 
the  Projective  Integration  of  Gear  and  Kevrekidis  [5].  In  the  projective 
integration  method  there  is  no  a  priori  knowledge  of  the  nature  of  the  coarse 
description.  In  their  coarse  projective  integration  method  it  is  assumed  that 
a  parametrization  of  the  slow  variable  is  given  and  that  it  can  be  computed 
from  the  observations. 

In  this  paper  we  consider  the  problem  of  integrating  systems  with  scale 
separation  where  the  separation  is  not  explicitly  given  in  the  equations  de¬ 
scribing  the  dynamical  system  and  hence  no  a  priori  knowledge  of  the  coarse 
description  is  at  hand.  Of  course,  if  additional  information  about  the  system 
is  given  in  advance,  one  should  use  it  to  make  the  computations  simpler  and 
faster. 

We  start  with  a  simple  example  of  our  formulation. 

HI.  There  exists  a  hidden  dynamical  system,  described  by, 

dx  =  y  (it 

dy  =  -(sinx  —  y)dt~\ — ^  d\\, 

€  y  6 

where 

1.  {x,y)  e  TZ  =  X  eTZx  =  y  =  and  t  G  [0,r]. 

2.  1. 

H2.  The  observables  of  the  system  (u,  u)  G  5  =  are  an  unknown,  smooth 
and  invertible  function  of  the  hidden  variables  x,  y  which  we  set  to  be 

2 


215 


B.  7.  CONSTRAINED  DYNAMICS  LIFTING 


F  :TZ  ^  S,  F(x,y)  =  {u,v), 


cos  ( II  (x,?/)  II)  .sin  ( II  (x,?/)  II) 
-sin(|j(x,?/)||)  cos(||(x,?/)||) 


)( 


X 


y 


•  (1.2) 


An  image  of  the  function  F  is  sketched  in  Figure  1. 

Using  results  from  the  averaging  principle,  we  have  the  following  approx¬ 
imation  which  we  now  describe.  We  start  by  defining  the  auxiliary  process. 
For  every  x, 


(1.3) 


If  we  ignore  the  noise  term  in  (1.1).  we  have  a  system  of  stiff  ODEs.  The 
auxiliary  process  (1.3)  corresponds  to  the  fast  equation  in  stiff  ODEs  when 


the  slow  variable  is  fixed.  The  next  step  after  fixing  the  slow  variable  when 


solving  stiff  ODEs  is  to  move  e  to  the  left  hand  side  and  to  set  is  to  zero. 
This  leads  to  an  algebraic  equation.  The  solution  of  the  algebraic  equation  is 
the  solution  at  time  equals  infinity  of  te  fast  dynamics  when  x  is  kept  fixed. 
Equivalently  it  is  the  limiting  position  of  the  y  variable  when  evolving  by  the 
fast  dynamic  equation  with  fixed  x.  When  the  auxiliary  process  is  stochastic 
one  follows  the  same  lines.  A  limiting  solution  of  the  auxiliary  process  is 
sought,  however  the  stochasticity  implies  that  the  limiting  solution,  when 
exists,  is  a  distribution  which  is  not  limited  to  a  stationary  position.  Basic 
knowledge  in  stochastic  processes  tells  us  that  in  our  example  the  auxiliary 
process  induces  a  stationary  measure  A/"(sinx,  1),  which  is  Guassian  with 
sin  X  as  its  mean  and  one  standard  deviation. 

Recalling  again  stiff  ODEs,  the  solution  of  the  algebraic  equation,  y  as 
a  function  of  x,  is  plugged  into  the  right  hand  of  the  equation  describing 
the  motion  of  the  slow  variable  to  give  in  our  example  dx  «  sinxdi.  In  the 
stochastic  case  we  average  the  right  hand  side  of  the  slow  dynamics  over  the 
stationary  measure  to  induce  the  averaged  dynamics  in  TZx  = 


dx  =  dt  J  yd{J\f{smx,l)}  =  dt  sirix. 

The  averaging  principle  asserts  that  for  t  G  [0,T). 

{x{t),y{t))  (x(0,F(oo))  ==  (x(0,V(sinx(i),l)). 


This  example  is  a  special  demonstration  to  the  case  where  the  knowledge 
of  y^(oo),  the  invariant  measure  generated  by  (1.3),  is  sufficient  to  generate 
x{t).  This  was  mainly  achieved  due  to  he  fact  that  one  knows  which  variables 
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(u,v)»F(x.y) 


> 


15  - 

1  ► 

0  5  ^ 

0' 

-0  5- 

-1- 

-19 - ^ - * - ' - ' - ^ 

-15  -1  HD  5  0  0  5  1 

U 


Figure  1:  The  image  of  the  non  linear  transformation  given  in  (1.2) 
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are  the  fast  variables  and  which  are  the  slow  variables.  In  all  other  cases 
which  we  categorized  under  Assumption  [H2].  there  is  no  known  scheme  that 
is  able  to  integrate  such  systems. 

This  section  is  organizes  as  follows.  In  Subsection  1.1  we  describe  the 
(coarse)  projective  integration  method.  In  Section  1.2  we  introduce  our 
algorithm. 

1.1  (Coarse)  Projective  Integration 

1.1.1  Start  with  projective  for  ODES 

Emphasize  that  the  slow  manifold  exists  because  of  the  enslaving.  This  is 
a  case  where  the  high  dimensional  evolution  reduces  to  a  lower  dimensional 
process.  Regardless  of  the  initial  position  of  the  particle,  it  takes  the  particle 
very  short  time  (which  is  usually  c  log  |)  to  approach  the  lower  dimensional 
manifold,  and  after  the  time  the  particle  evolves  only  close  to  the  lower 
dimensional  submanifold.  This  is  variable  reduction.  For  example  consider 

dx  =  ydt 

1  (1-4) 

dy  =  -(sill a:  —  y)  dt. 

If  we  start  at  (xo,yo)i  then  after  short  time  the  particle  will  be  positioned 
close  to  (xo,sinxo),  and  then  it  will  continue  evolving  along  the  curve 
{(x,  sinx)}  where  the  evolution  is  dictated  by  dx  =  siiixdt,  the  speed  at 
which  it  moves  along  the  curve. 

1.1.2  Move  to  projective  for  fast  stochastic  when  we  know  the 
slow  variable 

Explain  that  the  motion  is  in  the  entire  space,  the  particle  is  not  enslaved  to 
a  lower  dimensional  submanifold.  Instead  we  can  find  coarse  variables  that 
evolve  on  a  lower  dimensional  submanifold,  they  are  not  physical  particles. 

Consider  again  (1.1).  For  every  initial  value  (xo,yo)  the  particle  will 
explore  the  entire  y-space  before  we  can  see  a  change  in  the  x  direction. 
As  X  evolves  the  particle  will  continue  to  explore  the  entire  y-space  that 
corresponds  the  the  current  value  of  the  x  component.  What  the  averaging 
principle  gives  us  is  an  artificial  description,  in  the  sense  that  the  particle 
itself  doesn't  follow  this  description  but  rather  the  x-component  of  the  par¬ 
ticle  follows  that  reduced  evolution.  Note  that  in  our  examples  the  coarse 
description  is  the  same  both  for  the  stiff  ODEs  and  for  the  case  where  the  fast 
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dynamics  is  stochastic.  However,  we  emphasize  again  that  in  the  stochastic 
case  there  is  no  physical  particle  which  evolves  along  the  averaged  system. 

Conclude  with  the  difficulty  with  lifting/initilalizing  when  the  coarse 
variable  is  not  part  of  the  phase  space. 

1.2  Algorithm 

It  is  often  the  case  for  a  complex  system  that  one  is  usually  looking  for  a 
coarse  description,  we  do  the  same,  however,  the  algorithm  we  present  is 
independent  of  the  choice  of  the  coarse  variable.  A  predetermined  choice  of 
a  coarse  variable  doesn't  change  the  way  the  algorithm  is  implemented. 

The  averaging  principle  as  given  in  the  xy-space,  7^,  decouples  the  phase 
space  Ti  into  a  product  of  two  spaces  TZ  =  xR^,  where  the  first  subspace 
consists  of  only  the  subspace  where  the  slow  motion  occurs  and  the  second 
one  consists  of  only  the  subspace  where  the  fast  motion  occurs.  For  every 
X  G  R^"^,  the  fast  motion  induces  a  probability  distribution  jV(sinx,  1)  in 

.  This  is  equivalent  to  saying  that  in  TZ,  the  phase  space,  the  probability 
density  is  given  by, 

5(x)A7(sina’,  1). 

We  emphasize  that  although  in  TZ  there  exists  a  natural  coarse  variable 
and  natural  coarse  space  TZxi  ^  there  are  infinitely  many  “natural”  coarse 
spaces  and  coarse  variables.  To  see  this,  recall  our  system  (1.1).  The  fast 
dynamics  induces  a  Gaussian  measure  with  sin  x  as  its  mean,  and  the  natural 
slow  variable  will  evolve  in  R  like 


dx  =  sin  X  dt. 


However,  for  this  dynamical  system  an  important  coarse  variable  might  be 
the  evolution  of  the  mean  of  the  distribution  which  is  a  vector  in  R^  and  is 
given  by  {x{t)^smx{t)).  Hence  in  this  case  the  elimination  of  the  y  compo¬ 
nent  of  a  coarse  description  is  quiet  brutal. 

We  now  move  to  introduce  the  algorithm.  We  first  describe  it  using  hand 
waving  and  assuming  no  predetermined  coarse  variable  is  known. 

1.  Set  initial  value  in  the  observed  world 

2.  The  current  value  has  an  inverse  image  in  the  x,y  world  which  we 
denote  by  (x,  y).  Since  we  have  no  access  to  the  x,  y  world,  we  use  this 
only  as  a  fact. 
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3.  Evolve  the  dynamical  system  in  the  u^v  world  using  microscopic  time 
step  to  generate  a  good  empirical  sample  for  the  image  of  the  distri¬ 
bution  in  the  y  space  which  corresponds  to  the  current  x. 

4.  Calculate  the  median  of  the  empirical  sample.  The  median  will  be  our 
coarse  variable. 

5.  Evolve  the  system  for  a  period  of  Xt  where  6t  \t  At  and  record 
the  final  value. 

6.  Starting  at  that  value  evolve  the  dynamical  system  using  microscopic 
time  step  to  generate  a  second  good  empirical  sample  of  the  stochastic 
cloud. 

7.  Calculate  the  median  of  the  empirical  sample.  The  is  the  second  me¬ 
dian  point  for  evaluating  the  derivative  of  the  median. 

8.  Make  a  Projective  step  using  the  two  medians. 

9.  If  (n  <  A^)  repeat  the  procedure,  otherwise  END. 

In  order  to  move  to  the  more  formal  description  of  the  algorithm,  we 
now  define  the  symbols  we  use  throughout  the  algorithm: 

•  Let  d  :  5  X  5  — >  be  the  metric  in  S.  This  metric  is  used  for 
determining  the  medians.  This  knowledge  can  be  used  as  variance 
reduction  for  the  empirical  median.  However  in  this  paper  we  do  not 
address  this  point  any  further 

•  Define  Px  :TZ  to  be  the  function  (“projection”)  Px{x,y)  =  x 

•  Let  {uq,vo)  €  <S  be  an  initial  value 

•  {Un,Vn)  are  the  numerical  values  of  the  coarse  description 

•  Let  V^)  be  the  empirical  samples  of  a  cloud 

•  Recall  F(x,y)  =  (u,u)  to  be  the  unknown  function  between  the  two 
worlds 

•  for  i  =  l(resp.  i  =  2)  \s  the  first  (resp.  second)  median  used  for 
the  projective  step. 

Equipped  with  the  definitions  we  present  the  algorithm. 
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Algorithm: 

1.  Set  n  =  0,  2  =  1  and  =  (txoj^o)- 

2.  Given  (t/^  V^)  there  exist  (X,  Y)  = 

3.  Evolve  the  dynamical  system  using  microscopic  time  step  6t  to  gen¬ 
erate  m  =  1, . . . ,  A/,  where  M  is  large  enough  to  have  a 

good  empirical  sample  of  F((5(X)poo(y;  X)),  but  small  enough  to  insure 

4.  Set  Z;.  =  miiio<,<A/ 

5.  If  (i  =  1) 

(a)  {Un.Vn)  =  Z}, 

(b)  Evolve  the  system  for  a  period  of  Xt  where  St  Xt  At  and 
record  the  final  value  as  ([/^,  l^^). 

(c)  Set  i  =  2. 

(d)  Go  to  2. 

6.  Projective  step: 

(a)  {Un+uVn+l)  =  {Un,Vr,)  +  ^^At. 

(b)  i  =  1. 

(e)  If  (n  <  N)  go  to  2,  otherwise  END. 

The  coarse  description  that  we  have  constructed  is  given  by 

C-.=  {iUn,Vn)}^^,CS. 

We  have  chosen  {Un,  Ki)  =  to  be  the  median  [?]  point  of  the  experimental 
data  set  sampled  from  the  invariant  measure  F((5(X)poo(y;-^))- 

When  one  is  equipped  with  a  predetermined  coarse  variable  that  we  set 
to  be  Q  then  the  following  additions  need  to  be  made. 

Algorithm  with  predetermined  coarse  variable: 

1.  Set  n  =  0,  2  =  1  and  (C/^,  =  (uo,2;o). 

2.  Given  (f/^,  there  exist  (X,  Y)  =  V^). 
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3.  Evolve  the  dynamical  system  using  microscopic  time  step  St  to  gen¬ 
erate  {U^y  V^)  m  =  1, . . . ,  My  where  M  is  large  enough  to  have  a 
good  empirical  sample  of  F(5(X)/?oo(y;  X)),  but  small  enough  to  insure 

(a)  Set  Z*  =  mino<KM  Em=o  V‘)),  {U^,  K'")). 

(b)  Calculate  the  current  value  of  the  coarse  variable  and  denote  it 

as  Qi,. 

4.  If  {i  =  1) 

(a)  (Un,  Vn)  = 

(b)  Evolve  the  system  for  a  period  of  Xt  where  <C  At  -C  At  and 
record  the  final  value  as  (f/^,  V"^). 

(c)  Set  i  =  2. 

(d)  Go  to  2. 

5.  Projective  step: 

(a)  i.  {Un-^\yVri^\)  =  (UnyVn)  F 

ii-  Qn+l  =  Qn  +  At. 

(b)  i  =  1. 

(c)  If  (n  <  N)  go  to  2,  otherwise  END. 

The  coarse  description  that  we  have  constructed  is  given  by 

Q  •=  {Qn}„=o  ^ 

The  idea  in  this  algorithm  is  to  evolve  the  predetermined  coarse  variable 
using  the  standard  projective  schemes.  However  with  that  we  are  still  left 
with  the  problem  of  performing  the  lifting.  To  overcome  the  lifting  prob¬ 
lem  we  couple  the  standard  projective  scheme  with  the  new  algorithm  that 
evolves  the  medians.  In  such,  the  lifting  is  done  by  using  the  present  value 
of  the  median. 

2  The  same  thing  in  infinitesimal  generators  lan¬ 
guage 

The  relation  of  C  to  the  slow/coarse  variable  x  in  R  is  through  the  existence 
of  an  additional  smooth  and  invertible  function  G  such  that. 

F-1(C)=]F-1  ({([/„,  \/„)};)Lo)  CG(7^x). 
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We  start  by  formulating  the  problem. 

HI.  There  exists  a  hidden  dynamical  system,  described  by, 

dip{x,  y,  t)  =  Lp{x,  y,  t),  (2.1) 

where 

1.  (x,y)  €  7^  =  R",  X  G  Tlx  =  y  e  Tlj,  =  and  t  G  [O.T]. 

2.  ^ 

Zy  =  —  Z/1  -f*  Z/2* 

e 

3.  f  <  1. 

4.  the  derivatives  (or  differences  when  we  inelude  jumps)  prescribed 
in  Li  (resp.  L2)  are  only  in  the  y  (resp.  x)  direction. 

H2.  The  observables  of  the  system  (n,  i;)  G  5  =  are  an  unknown,  smooth 
and  invertible  function  of  the  hidden  variables  x,y  whieh  we  set  to  be 
F  :  7^  — ^  iS,  F{x,y)  =  (u,t;). 

Using  results  from  the  averaging  prineiple,  we  have  the  following  approxb 
mation, 

p{{x,y)  ,t)  ^  p{x,t)  poo{y\x).  (2.2) 

In  speeial  eases  when  the  evolution  of  the  slow  variable  ean  be  modeled  by 
a  differential  or  differenee  equations  whieh  ineludes  ODEs,  SDEs  and  Jump 
Proeesses,  the  knowledge  of  poo{y\^)  Is  suffieient  to  generate  p{x,t).  In  all 
other  eases  whieh  we  categorized  under  Assumption  [H2],  there  is  no  known 
scheme  that  is  able  to  integrate  such  systems. 

The  paper  is  organizes  as  follows.  In  Section  3  we  describe  the  (coarse) 
projective  integration  method.  In  Section  4  we  introduce  our  algorithm. 

3  (Coarse)  Projective  Integration 

3.1  Start  with  projective  for  ODES 

Emphasize  that  the  slow  manifold  exists  because  of  the  enslaving.  This  is 
a  case  where  the  high  dimensional  evolution  reduces  to  a  lower  dimensional 
process;  the  particle  evolves  only  on  a  lower  dimensional  submanifold.  This 
is  variable  reduction. 
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3.2  Move  to  projective  for  fast  stochastic  when  we  know  the 
slow  variable 

Explain  that  the  motion  is  in  the  entire  space,  the  particle  is  not  enslaved  to 
a  lower  dimensional  submanifold.  Instead  we  can  find  coarse  variables  that 
evolve  on  a  lower  dimensional  submanifold,  they  are  not  physical  particles. 

Conclude  with  the  difficulty  with  lifting/initilalizing  when  the  coarse 
variable  is  not  part  of  the  phase  space. 

4  Algorithm 

We  start  our  presentation  of  the  algorithm  for  a  simple  case  where, 

L2  =  a{x,y)  •  Vx, 

i.e.,  assuming  the  slow  variables  x  evolve  according  to  an  ODE, 

dx  =  a{x^y)  dt^ 

and  we  later  make  the  necessary  generalizations  for  more  complex  L2  gen¬ 
erators. 

It  is  often  the  case  for  a  complex  system  that  one  is  usually  looking  for 
a  coarse  description,  we  do  the  same,  however,  the  algorithm  we  present  is 
independent  of  the  choice  of  the  coarse  variable  and  in  some  sense  it  can 
be  used  as  a  ’’Coarse  Free  -  Equation  Free  ”  approach.  We  first  start  with 
exploring  the  submanifold  in  the  uu-space,  5,  which  is  the  image  of 

F(Supp(p 

DO 

where  Supp  is  the  support  of  the  density  in  TZ,  The  averaging  principle 
as  given  in  the  xy-space,  TZ,  decouples  the  phase  space  TZ  into  a  product 
of  two  spaces  TZ  =  x  where  the  first  subspace  consists  of  only 
the  subspace  where  the  slow  motion  occurs  and  the  second  one  consists  of 
only  the  subspace  where  the  fast  motion  occurs.  For  every  x  G  R^*,  the  fast 
motion  induces  a  probability  distribution  Poo{y\ ii^  This  is  equivalent 
to  saying  that  in  TZ,  the  phase  space,  the  probability  density  is  given  by, 

Pooi^.y)  :=  S{x)poo{y;x), 

We  emphasize  that  although  in  TZ  there  exists  a  natural  coarse  variable 
and  natural  coarse  space  =  TZx,  in  S  there  are  infinitely  many  “natural” 
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coarse  spaces  and  coarse  variables.  To  see  this,  consider  the  fast  dynamies 
to  be  described  by  an  OU  process. 

dx  =  ydt 

1  \/2 

dy  =  -{sin  X  —  y)  dt  -\ — ^  dB. 
e  y/e 

The  fast  dynamies  induces  a  Gaussian  measure  with  sin  x  as  its  mean,  and 
the  natural  slow  variable  will  evolve  in  R  like 

dx  =  sin  xdt. 

However,  for  this  dynamical  system  an  important  coarse  variable  might  be 
the  evolution  of  the  mean  of  the  distribution  Poo(^i  y)  which  is  in  Hence 
in  this  ease  the  elimination  of  the  y  component  of  a  coarse  description  is 
quiet  brutal. 

d  :  S  X  S  to  be  the  metric  in  S  and  ^  IRx  to  be  the 

function  (“projection’’)  Px{x^y)  =  x.  Let  (uo,i?o)  6  «S  be  an  initial  value. 

1.  Set  n  =  0,  i  =  1  and  {U^, 

2.  Given  (f/^  V^)  there  exist  (X,  Y)  =  V^). 

3.  Evolve  the  dynamical  system  using  microscopic  time  step  St  to  gen¬ 
erate  (f/^,  V^)  m  =  1, . . . ,  M,  where  M  is  large  enough  to  have  a 
good  empirical  sample  of  F(5(X)poo(2/;  X)),  but  small  enough  to  insure 

4.  Set  Z; 

5.  If  (i  =  l) 

(a)  {Un^Vn)  =  Z^ 

(b)  Evolve  the  system  for  a  period  of  Xt  where  St  ^  Xt  At  and 
record  the  final  value  as 

(e)  Set  i  =  2. 

(d)  Go  to  2. 

6.  Projective  step: 

(a)  (Un+uVn+l)  =  {Un,Vn)  + 

(b)  i=l. 
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(c)  If  (n  <  A^)  go  to  2,  otherwise  END. 

The  coarse  description  that  we  have  constructed  is  given  by 

C:={(t/n,V„)}toC5. 

We  have  chosen  {Un^  Vn)  =  to  be  the  median  [?]  point  of  the  experimental 
data  set  sampled  from  the  invariant  mea^iure  F(5(X)poo(^/;  A”)), 

The  relation  of  C  to  the  slow/coarse  variable  r:  in  7?.  is  through  the 
existenee  of  an  additional  smooth  and  invertible  funetion  G  sueh  that, 

F-1(C)  =  F->  ({(t/„,V„)};^^o)  CG(7^,). 

5  Generalizations 

In  this  section  we  give  different  generalization  to  our  novel  scheme.  Each 
generalization  corresponds  to  some  lack  of  a  priory  information  about  the 
dynamical  system  whieh  is  not  known  in  advanee.  The  less  information  we 
have  in  advance  about  the  system,  the  more  complexity  we  need  to  add  to 
the  solution. 

In  the  recent  years  ....  how  to  efficiently  parallel  the  algorithm 

5.1  Effective  dimension 

We  assume  here  that  we  do  not  know  in  advance  the  dimension  of  the  slow 
variables  and  the  dimension  of  the  fast  variables.  We  only  show  how  to  find 
the  dimension  of  the  fast  variables.  The  description  that  we  give  assumes 
that  e  is  at  the  limit  0. 

We  start  by  evolving  the  system  according  to  Step  3  in  the  algorithm  to 
generate  the  data  set  A  =  If  A  contains  no  acute  triangles 

then  .4  is  a  manifold  with  dimension  I,  U  A  contains  acute  triangles  but  no 
acute  three-dimensional  polyhedron  where  the  faces  are  acute  triangles,  then 
is  a  submanifold  of  dimension  2.  This  verification  can  continue  to  higher 
dimension  although  its  complexity  becomes  exponential  in  the  dimension. 

5.2  Hypothesis  Validation 

In  the  algorithm  we  presented  only  the  case  where  L2  =  a{x,y)  •  i.e., 

assuming  the  slow  variables  x  evolve  according  to  an  ODE,  dx  =  a{x,y)dt, 
Most  of  the  problems  that  appear  in  the  literature  in  the  context  of  scale 
separation  and  coarse  graining  can  be  categorized  into  three  types: 
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1.  The  coarse  variable  is  governed  by  an  ordinary  differential  equation 

2.  The  coarse  variable  is  governed  by  an  stochastic  differential  equation 
that  corresponds  to  a  Fokker-Planck  equation 

3.  The  coarse  variable  is  governed  by  a  jump  process  that  can  be  numer¬ 
ically  solved  using  Gillespie’s  SSA. 

Under  the  assumption  that  the  coarse  variable  is  either  one  of  these  types,  we 
now  present  the  generalization  of  our  original  algorithm  to  the  two  stochastic 
cases.  We  emphasize  that  we  do  not  need  to  know  in  advance  the  nature 
of  the  coarse  variable  and  we  can  verify  it  from  the  data.  We  start  by 
determining  to  which  category  the  dynamics  belong  and  then  we  show  how 
to  change  the  algorithm  to  incorporate  this  knowledge. 

1.  {U<^,V°)  =  {uo,vo). 

2.  Given  {U^,  there  exist  (X,  Y)  =  F-\U^,  V^). 

3.  Evolve  the  dynamical  system  using  microscopic  time  step  6t  to  gen¬ 
erate  (f/^,  V^)  m  =  1, . . . ,  A/,  where  M  is  large  enough  to  have  a 
good  empirical  sample  of  F((5(X)poo(2/;  AT)),  but  small  enough  to  insure 

4.  Set  Z"  =  mino<KM  Em=o  {U^,  V^)). 

5.  Repeat  for  1  <  A:  <  K, 

(a)  Set  to  be  initial  value 

(b)  Evolve  the  system  for  a  period  of  Xt  where  5t  Xt  At  and 
record  the  final  value  as 

(c)  Given  there  exist  {X,Y)  = 

(d)  Evolve  the  dynamical  system  using  microscopic  time  step  St  to 

generate  m  =  1,...,A/,  where  M  is  large  enough 

to  have  a  good  empirical  sample  of  F((5(X)/)oo(2/;  AT)),  but  small 
enough  to  insure  ^ 

(e)  Set  =  mino<KA/  Ei=o  dHU^  V^)),  ([/,-,  V^)- 

6.  Hypothesis  test: 

(a)  i,(Z«)  =  ^Ef=i  {Z'^-Z<^) 
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(b)  = 

With  that  we  can  test  to  get  the  following  results: 

1.  If  3A:  such  that  then  the  coarse  dynamics  is  governed  by  a 

jump  process  with  intensity  A  =  ).  check  whether  I  can  do 

even  the  jumps  or  just  approximate  them 

2.  If  ^42(2'^)  =  0  then  the  coarse  dynamics  is  deterministic. 

3.  In  all  other  cases  the  coarse  dynamics  is  governed  by  an  SDE. 

Now  that  we  know  the  nature  of  the  coarse  variable  we  can  change  the 
Projective  Step  in  the  algorithm.  We  present  here  the  algorithm  for  a  coarse 
variable  described  by  an  SDE: 

5.2.1  Algorithm  for  coarse  SDE 

1.  Set  n  =  0,  i  =  1  and  V^)  =  (uo,uo)- 

2.  Given  there  exist  (A,  F)  = 

3.  Evolve  the  dynamical  system  using  microscopic  time  step  6t  to  gen¬ 
erate  V^)  ni  —  1, . . . ,  A/,  where  M  is  large  enough  to  have  a 
good  empirical  sample  ofF{6{X)poo{y]  A)),  but  small  enough  to  insure 

4.  Set  =  mino<,<M  Em=o  V‘)),  ([/-,  V^)). 

5.  {{/„,V'„)  =  Z‘ 

6.  Repeat  for  1  <  A:  <  K, 

(a)  Set  Z^  to  be  initial  value 

(b)  Evolve  the  system  for  a  period  of  Xt  where  St  Xt  At  and 
record  the  final  value  as 

(c)  Given  ([/«,  V^)  there  exist  (X,  Y)  =  V^). 

(d)  Evolve  the  dynamical  system  using  microscopic  time  step  St  to 

generate  {UJP,V^)  m  =  1,...,A/,  where  M  is  large  enough 
to  have  a  good  empirical  sample  of  F(^(X)poo(y;  but  small 
enough  to  insure  P,  (F”*  ({C/"*,  V’”}i=o))  ^ 
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(e)  Set  =  mino<KM  Em=o  d{{Ui  V‘)),  {Ujp,  V,-)). 

7.  A,{Un,Vn)  =  jh^ELi 

8.  AliUn,  Vn)  =  ^  'Ek^i 

9.  Projective  step: 

(a)  (Un+l,Vn+l)  =  (Un,  V^)  +  A,{Un,  Vn)At  +  Vn)  dB. 

(b)  If  (n  <  N)  go  to  2,  otherwise  END. 

5.3  Coarse  projective 

In  this  section  we  discuss  the  case  where  we  axe  given  a  coarse  variable  D 
of  the  system  and  a  function  H  such  that  for  any  value  (u,  u)  €  «S  we  can 
calculate  D  =  H{u^v). 

The  problem 

6  Diffusion  maps  for  exploring  the  coarse  sub-manifold 

Clustering  and  low  dimensional  representation  of  high  dimensional  data  are 
important  problems  in  many  diverse  fields.  In  recent  years  various  spectral 
methods  [6]  to  perform  these  tasks,  based  on  the  eigenvectors  of  adjacency 
matrices  of  graphs  on  the  data  have  been  developed.  One  can  think  of  the 
diffusion  maps  as  a  way  to  revive  a  data  set  by  choosing  a  discrete  jump 
process  that  can  generate  the  same  data  set  as  its  stationary  distribution 
and  then  using  characteristics  of  the  jump  process  to  extract  information 
from  the  data  set. 

In  the  context  of  this  paper  and  under  appropriate  assumptions  one  can 
improve  the  way  the  jump  process  is  chosen.  If  the  coarse  variable  is  indeed 
restricted  to  TZxi  then  the  following  can  be  done. 

1.  Discretize  Tlx  label  the  space  segments  as  Set  a  zero 

matrix  =  0. 

2.  For  each  step  from  Bi  to  Bm  set  A{1,  m)  =  A{l,m)  -f- 1. 

3.  Normalize  the  rows  to  stochastic  matrix. 

A  is  now  an  approximation  to  the  operator  of  the  coarse  variable.  A{iyj) 
gives  the  transition  probability  between  the  zth  neighborhood  and  between 
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the  jth  neighborhood.  Now  with  this  matrix  one  can  continue  with  the 
following  parts  of  the  diffusion  maps  methods. 

When  the  above  assumptions  are  not  met,  for  example  in  the  case  where 
the  coarse  dynamics  is  not  restricted  to  IZx  rather  it  is  just  restricted  to  a 
submanifold  of  S,  then  the  use  of  the  diffusion  maps  method  is  very  appeal¬ 
ing  for  exploring  the  coarse  submanifold.  We  demonstrate  here  a  problem  in 
which  the  coarse  variable  evolves  according  to  a  double  well  FP  equation  on 
an  unknown  submanifold.  Since  the  dimension  of  the  phase  space  is  much 
larger  compared  to  the  dimension  of  the  submanifold  explored  by  the  coarse 
variable  the  discretization  suggested  above  becomes  inefficient. 
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Design  of  dynamics  for  self-assembly 

C.l  Fast  generation  of  potentials  for  self-assembly  of  particles 


231 


Cl.  FAST  GENERATION  OF  POTENTIALS  FOR  SELF-ASSEMBLY  OF  PARTICLES 


PRE/Self-Assonihly 

Fast  Generation  of  Potentials  for  Self  Assembly  of  Particles 

Philip  du  Toit,  Katalin  Grubits*,  and  Jerrold  Marsden 
Control  and  Dynamical  Systems,  California  Institute 
of  Technology,  MC  107-81,  Pasadena,  CA  9112^ 

Sorin  Costiner 

United  Technologies  Research  Center,  East  Hartford,  CT  06108 
(Dated:  April  9,  2009) 

Abstract 

\Vc  address  the  inverse*  problem  of  designing  isotropic  paim'isc  particle  interaction  potentials  that  lead 
to  the  forniatioii  of  a  desired  lattiec  when  a  system  of  particles  is  cooled.  The  design  problem  is  motivated 
l)y  the  desire  to  produce  materials  with  pre-sp(.*citicd  structure  and  properties.  We  present  a  heuristic 
coinputation-frcH*  geometric  method,  as  well  as  a  fast  and  robust  trend  optimization  method  that  lead  to 
the  formation  of  liigli  quality  lioneycomb  lattices.  The  trend  optimization  method  is  particularly  successful 
since  it  is  well-siiiteti  to  efficient  optimization  of  tlie  noi.sy  and  expensive  objective  functions  encountered  in 
the  sclf-assoiiibly  design  problem.  We  also  present  anisotropic  potentials  that  robustly  lead  to  the  formation 
of  the  kagoinc  lattice,  a  lattice  that  has  not  previously  been  obtained  with  isotropic  potentials. 
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I.  INTRODUCTION 

During  the  process  of  self-assembly,  randomly  distributed  components  acting  under  the  influence 
of  short-range  mutual  interactions,  arrange  themselves  into  a  highly  ordered  final  configuration. 
An  important  feature  of  self-assembly  is  that  the  components  arrange  themselves  into  this  more 
ordered  state  without  the  influence  of  external  factors.  The  ordered  arrangement  observed  at  the 
level  of  the  resulting  superstructure  is  predicated  on  the  design  of  the  individual  components  and 
their  local  interactions.  Understanding  the  self-assembly  process  and  how  local  properties  of  the 
comp>onents  may  be  manipulated  to  influence  the  resulting  global  ordering,  is  an  active  area  of 
research  spanning  a  broad  range  of  disciplines  including  materials  science,  chemical  engineering, 
bioengineering,  and  nanotechnology. 

In  a  seminal  article,  Whitesides  and  Grzybowski  [1]  provided  a  broad  definition  of  self-assembly, 
and  addressed  promising  applications  in  a  wide  range  of  disciplines.  They  observed  that  self- 
assembly  of  cells  to  form  tissue,  organs,  and  ultimately  organisms,  is  fundamental  to  life,  and 
motivates  a  determined  study  of  the  self-assembly  process.  Subsequent  studies  and  applications  of 
self-assembly  are  as  fascinating  as  they  are  plentiful.  Murr  et  al  [2]  identified  self-assembly  as  the 
mechanism  by  which  silicatein  monomers  combine  to  form  protein  fibers  in  certain  marine  sp>onges. 
Zheng  et  al  used  shape  recognition  and  selective  binding  to  induce  self-assembly  and  packaging 
of  integrated  semiconductor  microsystem  devices  by  agitating  the  components  in  an  aqueous  en¬ 
vironment  [3],  and  Stauth  et  al  [4]  have  manufactured  field-effect  transistors  via  self-assembly  of 
micrometer  scale  components  using  similar  techniques.  In  [5],  self-assembly  is  shown  to  enhance 
synergistic  group  transport  of  autonomous  robots,  Jakab  et  al  recognized  that  although  biolog¬ 
ical  systems  are  genetically  controlled,  the  formation  of  biological  superstructures  is  ultimately 
governed  by  physical  interactions,  and  demonstrated  the  formation  of  prescribed  shapes  using 
self-assembling  multicellular  systems  [6]. 

Typically,  studies  of  self-assembly  examine  the  ordered  superstructures  that  arise  from  a  given 
fixed  interaction  potential.  For  example,  Manoharan  et  al  used  optical  and  electron  microscopy 
to  identify  the  range  of  structures  produced  by  self-assembly  of  colloidal  microspheres  as  fluid  is 
removed  from  the  emulsion  droplets  containing  the  spheres  [7].  Maksymovych  et  al  investigated 
the  formation  and  reactivity  of  linear  chains  of  dimethyldisulfide  molecules  on  a  gold  surface  [8j. 
Engel  et  al  ([9],  see  also  [10-13])  have  observed  the  formation  of  complex  crystals  and  quasicrystals 
arising  from  a  simple  double-well  interaction  potential. 

Interest  in  the  fabrication  of  nanomaterials  and  photonic  crystals  with  desired  material  prop- 
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erties  [14  21]  motivates  the  inverse  problem:  design  the  shortrange  interactions  in  order  to  in¬ 
duce  self-assembly  of  the  components  into  a  desired  lattice  structure.  Laboratory  techniques  now 
available  allow  for  modification  and  tuning  of  particle  interaction  potentials  [22],  and  hence  ex¬ 
perimentalists  are  achieving  ever-increasing  control  over  the  local  interactions  that  influence  the 
global  properties  of  the  material  formed  via  self-assembly.  These  methods  typically  use  colloidal 
suspensions  and  optical  forcing  to  alter  the  chemical  environment  and  screening  properties  of  the 
solution  in  which  the  assembly  occurs. 

In  this  paper,  we  specifically  consider  the  problem  of  designing  short-range  pairwise  interaction 
potentials  between  particles  on  a  planar  surface  to  induce  self-assembly  of  a  desired  lattice.  The 
main  results  to  be  presented  are  new  methods — a  heuristic  geometric  method  as  well  as  a  robust 
trend  optimization  method  for  the  design  of  isotropic  interaction  potentials  that  lead  to  high  quality 
honeycomb  lattices  as  the  system  of  particles  is  cooled.  The  geometric  method  is  also  extended  to 
the  case  of  anisotropic  potentials  which  allows  for  the  formation  of  more  exotic  kagome  lattices. 
Another  contribution  of  this  work  is  the  development  of  tools  for  objective  assessment  of  quality 
of  lattices,  that  mimic  intuitive  human  perception  of  lattice  quality. 

Rechtsman  et  al  in  [23]  have  already  demonstrated  computational  methods  for  finding  solutions 
to  the  inverse  self-assembly  problem.  They  used  a  simulated  annealing  optimization  procedure  to 
find  potentials  that  lead  to  the  self-cissembly  of  particles  into  square  and  honeycomb  lattices.  To 
be  sure,  the  intent  of  [23]  was  to  demonstrate  that  the  inverse  problem  of  potential  design  for  the 
purpose  of  inducing  the  formation  of  a  target  lattice  can  be  solved  in  practice,  and  to  carefully 
verify  that  the  potentials  they  proposed  do  indeed  lead  to  the  target  lattices  through  Monte  Carlo 
simulation.  Hence,  the  computational  effort  required  to  obtain  the  potentials  was  only  a  marginal 
consideration  in  their  work.  Presently,  we  consider  the  straightforward  simulated  annealing  method 
of  [23]  as  a  baseline  method  with  which  we  may  compare  the  new  methods  presented  here.  When 
compared  with  the  baseline  simulated  annealing  method,  the  optimization  procedure  described  in 
this  paper  leads  to  a  hundredfold  speed-up  in  the  generation  of  potentials,  as  well  as  the  formation 
of  higher  quality  lattices.  Furthermore,  the  procedure  for  finding  potentials  is  more  robust,  and  the 
resultant  potentials  form  the  target  lattices  more  robustly  with  respect  to  variations  in  the  initial 
conditions  of  the  particles.  As  will  be  demonstrated,  the  chief  reason  for  the  marked  speed-up 
over  the  simulated  annealing  method  is  the  facility  of  the  trend  optimization  method  to  optimize 
objective  functions  that  are  both  noisy  and  expensive  to  evaluate. 

The  organization  of  the  paper  is  as  follows.  In  Section  II.  we  precisely  define  the  self-assembly 
problem,  and  summarize  the  method  for  generating  potentials  devised  previously  by  [23]  that  will 
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serve  as  a  baseline  method  for  purposes  of  comparison.  In  Section  111,  we  establish  objective  metrics 
for  measuring  lattice  quality  so  that  reasonable  comparisons  between  the  methods  can  be  made. 

We  approach  the  self-assembly  problem  by  framing  it  as  an  optimization  problem  in  which 
the  desired  potential  optimizes  a  suitably  chosen  objective  function.  This  approach  requires  that 
we  choose  both  an  optimization  scheme  and  an  objective  function  to  be  optimized.  Section  IV 
describes  three  objective  functions  that  will  be  used,  while  Section  V  describes  in  detail  the  trend 
optimization  method.  A  discussion  on  the  hierarchical  nature  of  the  trend  method  is  also  provided. 
We  proceed  in  Section  VI  to  list  the  five  solution  methods  to  be  compared  and  describe  their 
implementation.  The  final  comparison  of  the  methods  is  presented  graphically  in  the  plots  of 
Section  VIL  In  Section  VIII,  we  present  an  extension  of  the  geometric  method  to  anisotropic 
potentials  that  lead  to  self-assembly  of  the  kagome  lattice. 

All  molecular  dynamics  simulations  performed  during  the  design  and  testing  of  the  potentials 
were  executed  on  the  CITerra  high  performance  computing  cluster  housed  in  the  Division  of  Geo¬ 
physical  and  Planetary  Sciences  at  Caltech  using  the  LAMMPS  software  package  [24]  from  Sandia 
Laboratories. 
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IL  THE  SELF  ASSEMBLY  PROBLEM 

The  availability  of  laboratory  methods  to  tune  interaction  potentials  between  components,  and 
hence  influence  the  structure  of  the  resulting  self-assembled  configurations,  motivates  the  use  of 
self-assembly  to  produce  materials  with  desired  structural  properties.  The  specific  self-assembly 
problem  addressed  in  this  paper  is  defined  as  follows: 

Definition:  The  Self-Assembly  Problem 

Design  a  radially  symmetric  pairwise  interaction  potential,  V^hc(^)j  that  when  a 

system  of  particles  interacting  with  each  other  in  the  plane  through  this  potential  is 

cooled,  the  particles  form  a  honeycomb  lattice. 

The  purpose  of  the  present  paper  is  to  compare  methods  for  generating  potentials  that  solve  the 
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self-aife'embly  problem.  The  methods  for  generating  potentials  are  compared  using  three  criteria: 

Cl.  The  computational  effort  required  by  the  method  to  produce  the  potential; 

C2.  The  quality  of  the  lattices  formed  by  the  potential; 

C3.  The  robustness  of  the  quality  of  the  lattices  formed  to  variations  in  the  initial  conditions  of 
the  particles. 

Certainly,  for  a  given  method  we  expect  to  see  trade-offs  between  these  criteria.  For  example, 
a  faster  method  may  lead  to  a  potential  that  produces  lattices  of  poorer  quality. 

Laboratory  techniques  for  tuning  interaction  potentials  motivated  Rechtsman  et  al  [23]  to  con¬ 
sider  physically  realizable  potentials  as  basis  functions  for  the  desired  potential,  where  the  basis 
functions  contain  parameters  that  allow  for  tuning  the  shape  of  the  total  potential  obtained  from 
their  sum.  For  the  case  when  the  desired  final  configuration  of  particles  is  the  honeycomb  lattice, 
[23]  proposed  an  interaction  potential  consisting  of  the  sum  of  a  Lennard-Jones  potential,  an  ex¬ 
ponentially  decaying  potential,  and  a  Gauss ian-shaped  potential,  parameterized  in  the  following 
way: 

VHc(r;  00,01,02,  as)  =  ^  -  ^  +  “i  6“°”'  -  0.4  (1) 

where  oq,  ai.  02,  and  03  are  four  free  parameters  that  can  be  tuned  to  adjust  the  shape  of  the 
potential. 

A  solution  (there  are  many)  to  the  self-assembly  problem  has  been  provided  by  [23]:  namely, 
choosing  oq  =  5.89,  aj  =  17.9,  a2  =  2.49,  and  03  =  1.823  in  the  expression  for  Vhc-  A  sample  lattice 
obtained  when  cooling  a  system  of  particles  using  these  parameters  for  the  interaction  potential 
is  shown  in  Figure  1(c).  The  honeycomb  lattice  is  the  dominant  structure  in  the  lattice,  although 
there  are  still  visible  defects  that  arise  due  to  the  finite  duration  of  the  cooling  schedule.  The 
cooling  simulation  used  to  obtain  this  lattice,  as  well  as  all  other  simulations  referred  to  in  this 
paper,  were  performed  using  periodic  boundary  conditions. 

A  reasonable  question  to  ask  at  this  point  is:  **How  difficult  is  the  self-assembly  problem?” 
Experience  shows  that  although  easy  to  state,  the  self-assembly  problem  is  difficult  to  solve  in  that 
solutions  that  lead  to  the  honeycomb  lattice  are  difficult  to  find  and  possibly  very  fragile.  For 
instance,  adding  small  perturbations  to  the  parameters  in  [23]  for  the  honeycomb  potential  leads 
to  the  configuration  in  Figure  2(b)  in  which  the  honeycomb  structure  is  less  pronounced.  The 
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(b)  The  potential  produced  by  [23].  (c)  A  sample  lattice  obtained 

using  the  potential  provided  by 
[23). 


FIG.  1 :  The  self-assembly  problem  entails  finding  an  interaction  potential  that  leads  to  the  formation  of  a  honeycomb 
lattice  (a).  [23]  used  a  simulated  annealing  optimization  procedure  to  generate  a  potential  (b).  A  sample  lattice 
formed  using  this  potential  is  shown  in  (c)  and  exhibits  defects  that  result  from  the  finite  duration  of  the  cooling 
simulation. 


(a)ao  =  5.89,  Oi  =  17.9,  (b)  Oo  =  5.89,  Oi  =  17.9,  (c)  Oo  =  5.0,  ai  =  17.0,  (d)  Oo  =  6.0,  ai  =  18.0, 

02  =  2.49,  0.1  =  1.823.  02  =  2.49,  o.i  =  1.89.  02  =  2.0,  0.1  =  1.5.  02  =  3.0,  03  =  2.0. 


FIG.  2:  The  lattices  shown  here  are  obtained  by  slowly  cooling  484  particles  using  the  interaction  potential 
VHc(r;o«, 01,02, 03)  provided  in  Equation  1  with  the  parameter  values  indicated.  Figure  2(a)  uses  the  parameters 
proposed  by  [23],  while  Figure  2(b)  indicates  how  fragile  the  honeycomb  lattice  is  with  respect  to  slight  changes  in 
the  parameters.  Figures  2(c)  and  2(d)  indicate  the  range  of  structures  obtained  for  generic  values  of  the  parameters. 


amorphous  configurations  shown  in  Figures  2(c)  and  2(d)  indicate  the  range  of  structures  that  are 
possible  for  generic  values  of  the  parameters. 

We  have  identified  the  honeycomb  lattice  as  the  target  lattice  in  the  self-assembly  problem 
because  it  the  simplest  lattice  structure  that  represents  a  non-trivial  case  for  self-assembly.  The 
triangular  lattice  is  easily  and  robustly  assembled  by  a  straightforward  Lennard-Jones  interaction 
potential.  This  fact  can  be  readily  demonstrated  through  direct  simulation;  however,  it  is  interest- 
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ing  that  a  rigorous  mathematical  proof  that  the  triangular  lattice  is  the  global  energy  minimizer 
and  ground  state  of  a  particle  system  with  a  pairwise  Lennard-Jones  interaction  potential  was 
provided  only  recently  by  Theil  in  [25].  Generating  a  potential  that  robustly  induces  the  formation 
of  a  square  lattice  in  particle  simulations  is  also  a  simple  matter  using  the  geometric  method  to  be 
presented  later.  Producing  the  honeycomb  lattice  using  only  a  radially  symmetric  potential  is  more 
troublesome  chiefly  because  regions  of  the  domain  tend  to  form  the  triangular  lattice  -  a  lattice 
that  competes  strongly  with  the  honeycomb  lattice.  This  is  due  to  the  fact  that  the  triangular 
and  honeycomb  lattices  have  identical  distances  to  nearest  neighbors,  and  the  triangular  lattice  is 
formed  by  simply  adding  a  particle  to  the  center  of  each  hexagonal  cell  in  the  honeycomb  lattice. 

For  concreteness,  we  constrain  the  parameter  space  over  which  we  search  for  parameter  values 
in  FHc(naojQi)  02,03)  such  that 


4.0  <  ao  <  8.0,  0  <  oi  <  30.0,  0  <  02  <  3.8,  and  1.25  <  03  <  2.25.  (2) 


These  ranges  yield  a  large  class  of  function  shapes  over  which  the  search  is  performed. 


Rechtsman  and  co-workers  developed  two  computational  algorithms  to  find  potentials  that  lead 
to  the  self-assembly  of  particles  into  a  given  target  lattice  [23,  26]. 

The  first  optimization  scheme  chooses  the  shape  of  the  potential  so  that  the  energy  difference 
between  the  target  lattice  configuration  and  the  configuration  of  the  competitor  lattices  is  max¬ 
imized.  The  optimization  is  performed  while  ensuring  mechanical  stability  by  allowing  only  real 
phonon  frequencies.  This  method  is  purely  static  in  that  it  seeks  to  ensure  that  the  target  lat¬ 
tice  configuration  is  energetically  the  most  preferred  final  state;  the  method  does  not  incorporate 
information  about  the  dynamics  of  the  particles  as  they  tend  towards  this  final  configuration. 

The  second  optimization  scheme  considered  by  Rechtsman  et  al  concentrates  on  choosing  poten¬ 
tials  so  as  to  maximize  the  stability  of  the  target  lattice  near  its  melting  point,  while  also  requiring 
stability  of  the  the  target  lattice  with  respect  to  changes  in  density,  and  mechanical  stability  by 
ensuring  that  the  phonon  frequencies  are  real.  After  placing  particles  into  the  target  lattice  con¬ 
figuration,  a  short  molecular  dynamics  simulation  is  performed  at  a  fixed  temperature  just  below 
the  melting  temperature  of  the  lattice.  The  deviation  of  the  the  final  configuration  from  the  initial 
target  lattice  is  computed  using  the  Lindemann  parameter. 


(3) 


where  N  is  the  number  of  particles,  and 


and  Ti  are  the  initial  and  final  positions  of  particle 
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respectively.  A  simulated  annealing  procedure  is  used  to  choose  parameters  in  the  expression  for 
VHc(n  <13)  that  minimize  the  Lindemann  parameter. 

After  generating  a  potential  using  these  methods,  Rechtsman  et  al  carefully  checked  using  Monte 
Carlo  simulations  that  particles  starting  in  a  random  initial  configuration  do  indeed  self-assemble 
into  the  target  lattice.  The  determination  as  to  whether  or  not  the  target  lattice  was  formed, 
was  made  by  visual  inspection  of  the  final  configuration  and  deciding  if  the  configuration  has  few 
enough  defects  to  be  considered  a  lattice,  as  well  as  checking  the  long  range  order  by  visually 
inspecting  simulated  Bragg  diffraction  patterns. 

III.  LATTICE  QUALITY  METRICS 

As  described  in  Section  II,  one  of  the  criteria  used  to  compare  methods  for  generating  interaction 
potentials  is  the  quality  of  the  resulting  lattices.  In  the  work  of  [23],  a  simple  visual  check  of  the 
resulting  lattice  was  used  to  determine  if  the  desired  lattice  (with  a  few  defects  perhaps)  was 
obtained,  and  since  their  purpose  was  to  show  that  the  desired  lattices  can  be  obtained,  this  visual 
check  was  sufficient.  The  purpose  of  the  present  paper  is  to  compare  several  methods  according  to 
the  lattice  quality  criterium,  thus  we  must  first  introduce  objective  methods  for  assessing  lattice 
quality. 

A  prevalent  metric  for  lattice  quality  is  the  structure  factor,  a  quantity  that  assesses  long  range 
spectral  order  in  the  lattice  using  diffraction  patterns.  We  have  found  that  this  quality  metric  is 
inadequate  for  our  purposes,  firstly  because  the  structure  factor  does  not  provide  a  scalar  value  for 
quality,  and  secondly  because  we  have  observed  that  long  range  order  is  not  necessarily  strongly 
correlated  with  visual  perception  of  lattice  quality  -  a  lattice  with  glaring  defects  may  still  exhibit 
a  high  degree  of  long  range  order,  for  example,  while  a  lattice  that  has  weak  long  range  order 
due  to  a  grain  boundary  may  have  excellent  local  lattice  structure.  Hence,  we  have  developed  two 
lattice  quality  metrics  that  mimic  nearly  as  possible  the  visual  assessment  of  lattices.  When  the 
human  eye  looks  at  a  lattice  and  makes  a  judgement  with  respect  to  lattice  quality,  the  emphasis 
is  on  order  within  sub- regions  of  the  entire  lattice.  A  lattice  that  consists  of  two  perfectly  formed 
sub-lattices  that  have  a  domain  wall  where  they  meet  will  be  judged  by  the  eye  to  be  quite  well- 
formed.  Thus,  although  long  range  order  is  important,  a  determination  of  local  ordering  is  crucial 
for  assessing  lattice  quality  in  a  manner  similar  to  the  eye.  The  metrics  that  we  use  to  quantify 
lattice  quality  have  this  local  feature. 

The  two  lattice  quality  metrics  we  present  here  are  called  the  Template  Measuir  and  the  Defect 
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Measure,  In  both  cases,  a  lower  value  of  the  metric  corresponds  to  a  higher  quality  lattice. 


A.  Template  Measure 


The  Template  Measure  uses  a  small  segment  of  the  target  lattice  as  a  template  with  which 
to  locally  compare  nearby  lattice  particle  positions  for  each  particle  in  the  given  lattice.  In  the 
honeycomb  lattice,  a  suitable  template  may  be  one  hexagonal  cell  composed  of  six  particles,  or 
p)oints.  For  each  particle  in  a  given  lattice  configuration,  the  first  point  in  the  template  is  pinned 
to  the  particle,  and  the  template  is  then  rotated  to  find  the  best  fit  to  other  nearby  particles. 
As  the  template  is  rotated,  each  point  in  the  template  is  paired  with  the  nearest  particle  in  the 
given  lattice.  The  angle  of  rotation  of  the  template  that  produces  the  least  deviation  between  the 
template  points  and  lattice  particles  is  considered  the  best  fit.  Once  this  best  fit  position  of  the 
template  has  been  found  for  each  particle  in  the  lattice,  the  Template  Measure  (TM)  is  obtained 
by  summing  the  deviation  in  the  positions  of  the  template  points  and  lattice  particles  from  the 
best  fit  for  each  particle  in  the  lattice: 


TKA  • 

TM  :=  >  mm 

N  ^  9 

p^\ 


(J  ^  0,tcmplatc\^ 

2^  ^i,p  j  T  ^p,e 

t=2 


(4) 


where  the  index  p  ranges  over  all  N  particles  in  the  given  lattice,  9  is  the  angle  of  rotation  of  the 
template,  is  the  position  of  the  point  in  the  template  when  the  template  is  attached 

to  particle  p  and  rotated  by  angle  0,  rfp  is  the  position  of  the  particle  in  the  given  lattice  that 
is  closest  to  and  c  is  the  number  of  points  in  the  template  (c  =  6  for  the  honeycomb 

cell  template).  Notice  that  since  the  first  point  in  the  template  is  pinned  to  the  given  lattice 
particle  their  positions  are  equal,  that  is  p  =  and  hence  the  sums  over  the  template 

points  need  not  consider  this  first  template  point.  The  extra  term.  Hp, extra »  is  a  count  of  any  extra 
particles  in  the  given  lattice  that  fall  inside  the  hexagonal  template,  but  are  not  paired  with  any  of 
the  template  points.  In  this  way,  the  best  fit  of  the  template  seeks  not  only  to  match  the  particle 
locations,  but  also  the  void  within  the  hexagon.  This  ‘opacity’  of  the  template  ensures  that  defects 
that  arise  due  to  the  formation  of  the  triangular  lattice  that  has  a  particle  located  at  the  center  of 
the  hexagon  will  be  penalized.  The  prepended  scaling  factor  of  100 /N  is  not  strictly  necessary  in 
the  Template  Measure,  but  is  included  for  convenience  so  that  the  resulting  lattice  quality  values 
can  be  interpreted  as  a  measure  of  the  defectiveness  per  particle,  and  have  a  magnitude  in  a  range 
from  zero  to  roughly  100. 

An  illustration  of  how  the  Template  Measure  is  implemented  in  practice  is  provided  in  Figure  3. 
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In  Figure  3(a),  a  single  hexagonal  honeycomb  cell  template  is  attached  to  a  particle  in  a  honeycomb 
lattice  and  rotated  until  a  best  fit  with  the  surrounding  particles  is  achieved.  Repeatedly  attaching 
the  template  to  each  particle  in  the  lattice  and  realigning  as  in  Figure  3(b)  quickly  reveals  the 
locations  of  the  defects. 


(a) 


(b) 


FIG.  3:  Illustration  of  the  Template  Measure,  (a)  A  template  eonsisting  of  a  single  hexagonal  eell  is  pinned  to  a 
particle  in  the  given  lattiee,  and  rotated  to  the  position  whieh  minimizes  the  distanee  between  points  in  the  template 
and  the  nearest  particles  in  the  lattiee.  (b)  Fitting  the  template  cell  to  each  particle  in  the  lattice  and  rotating  to 
find  the  best  fit,  quickly  reveals  the  locations  of  defects. 


B.  Defect  Measure 

The  second  lattice  quality  metric  we  present  is  the  Defect  Mexisure.  The  Defect  Measure  provides 
a  weighted  count  of  all  the  defects  in  the  local  neighborhood  of  each  particle  in  the  given  lattice. 
The  types  of  defects  considered  are  shown  in  Figure  4,  and  include  displciced,  missing,  and  extra 
particles.  Note  that  all  of  the  possible  defects,  including  global  defects,  are  taken  into  account  by 
the  types  of  defects  shown.  For  example,  extended  grain  boundaries  are  taken  into  account  by 
contributions  to  the  Defect  Measure  from  locally  displaced,  missing,  and  extra  particles. 

Recall  that  in  the  perfect  honeycomb  lattice,  the  distance  to  nearest  neighbors  is  unity,  while 
the  distance  to  second  nearest  neighbors  is  \/3.  In  order  to  calculate  the  Defect  Measure,  we 
consider  only  a  small  circular  region  around  each  particle  of  radius  (1  -h  \/3)/2  ^  1.366  so  that 
only  three  nearest  neighbors,  each  at  unit  distance,  should  be  included.  Then,  using  the  locations 
of  all  particles  actually  located  within  this  circular  region  for  the  given  lattice,  the  contributions 
from  each  of  the  defects  is  calculated,  and  then  summed  with  the  speci6ed  weighting,  for  each 
particle  in  the  given  lattice  to  provide  a  measure  of  the  quality  of  the  lattice  as  a  whole.  The 
weights  attributed  to  each  type  of  defect  may  be  chosen  according  to  the  desired  properties  for  the 
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FIG.  4:  To  compute  the  Defect  Measure,  the  number  of  defects  in  a  circular  neighborhood  of  each  particle  arc 
weighted  and  summed.  The  types  of  defects  used  in  computing  the  Defect  Measure  are  displaced,  missing,  and  extra 
particles,  as  well  as  lone  and  boundary  p>articles. 


self-aascmbled  lattice.  These  weights  may  emphasize  correct  local  densities,  correct  alignment  of 
particles,  correct  inter-paxticle  distances,  or  any  other  customized  weighting. 

Accordingly,  the  Defect  Measure  (DM)  is  given  by 
N 


DM  :=  f  E 


P=1 


^displaced  I  x(^j)  •  {(^j  10)^ 


-H  w^missing  *  ^p, missing  "h  ^extra  *  ^p, extra  “h  ^onc  *  ^p,lonc  “h  i^boundary  *  ^p,boundary 

where  the  a;’s  are  the  weights  attributed  to  each  type  of  defect;  the  rip’s  are  the  integer  number 
of  missing  and  extra  particles  within  the  small  circular  region  surrounding  particle  p;  the  r^p’s  arc 
indicator  functions  that  equal  unity  if  particle  p  is  a  lone  or  boundary  particle  and  zero  otherwise; 
the  index  p  ranges  over  all  the  N  particles  in  the  given  lattice;  the  index  j  ranges  over  the  particles 
contained  within  the  small  circular  region  surrounding  particle  p;  and  dpj  is  the  positive  distance 
between  particle  p  and  particle  j.  As  with  the  Template  Measure,  the  scaling  factor  of  100/A^  is 
included  so  that  the  quality  values  are  normalized  by  the  number  of  particles  and  fall  within  a 
convenient  range.  In  the  first  term,  y(-)  is  a  smooth  cutoff  function  that  decreases  from  1  to  0 
at  the  outer  edges  of  the  circular  region,  or  more  precisely,  as  its  argument  increases  from  1.275 
to  1.366.  This  smooth  cutoff  function  ensures  that  the  Defect  Measure  remains  continuous  with 
respect  to  motion  of  the  particles  and  as  the  number  of  particles  entering  the  circular  region  of 
each  particle  fluctuates. 
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More  explanation  of  the  Defect  Meaijure  and  its  applications  can  be  found  in  [27],  where  there 
is  also  a  discussion  of  other  lattice  quality  metrics. 

For  the  purposes  of  this  paper,  we  use  the  following  w’eight  values  for  the  Defect  Measure: 


^displaced  —  1*0,  ^^missing  —  1*5,  ^extra  —  0.8,  ^^onc  —  2.0,  ^boundary  —  0.1. 

Sample  values  of  both  the  Template  Measure  and  the  Defect  Measure  for  four  lattices  of  varying 
quality  are  shown  in  Figure  5. 


(a)TM  =  131.6 
DM  =  161.3 


DM  =  42.1 


DM  =  14.9 


S 


(d)TM  =  19.9 
DM  =  9.6 


(c)TM  =  10.2 
DM  =  1.6 


FIG.  5:  Sample  values  of  the  lattice  quality  metrics  arc  shewn  for  a  range  of  lattices.  In  each  figure,  the  value  of 
the  Template  Measure  (TM)  and  the  Defect  Measure  (DM)  are  provided.  Both  measures  assign  a  lower  value  to  a 
lattice  of  higher  quality. 


IV.  OBJECTIVE  FUNCTIONS 


A  natural  way  in  which  to  find  solutions  to  the  self-assembly  problem  is*  to  search  for  a  potential 
that  optimizes  an  appropriate  objective  function.  The  objective  function  must  be  well-chosen  so 
that  1)  optimizing  the  objective  correlates  w’ell  w’ith  the  formation  of  the  desired  lattice,  and  2) 
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the  objective  function  is  not  prohibitively  expensive  to  evaluate.  In  this  section,  we  present  the 
objective  functions  that  will  be  used  in  the  potential  generation  methods  under  comparison. 

A.  The  Lindemann  Parameter  Objective  Function 

As  noted  in  Section  11,  the  Lindemann  parameter  is  a  measure  of  how  much  a  lattice  configu¬ 
ration  has  deviated  from  the  perfect  lattice  configuration  after  a  short  simulation.  Consequently, 
the  Lindemann  parameter  cannot  be  used  as  a  metric  for  lattice  quality  per  se  since  it  measures 
how  far  a  lattice  has  deviated  when  initially  placed  in  the  perfect  target  lattice  configuration, 
whereas  a  lattice  quality  metric  must  assess  the  quality  of  a  lattice  obtained  after  a  slow  cooling 
process  from  an  arbitrary  initial  condition.  The  Lindemann  parameter  can  nevertheless  be  used 
as  an  objective  function  that  must  be  minimized  in  order  to  find  potentials  that  lead  to  formation 
of  the  target  lattice  -  an  approach  that  was  first  proposed  and  developed  by  [23].  Intuitively, 
minimizing  the  Lindemann  parameter  produces  potentials  that  stabilize  the  honeycomb  lattice  to 
thermal  agitation.  Used  as  an  objective  function,  the  Lindemann  parameter  is  advantageous  since 
it  is  much  faster  to  evaluate  than  other  objective  functions  that  require  much  longer  molecular 
dynamics  simulations. 

As  implemetcd  in  this  paper,  the  Lindemann  parameter  objective  function  is  computed  for  a 
given  interaction  potential  by  placing  72  particles  in  the  honeyomb  lattice  formation,  and  then 
performing  a  brief  simulation  at  a  temperature  very  near  the  melting  temperature  of  the  lattice. 
The  use  of  72  particles  allows  for  the  construction  of  a  lattice  consisting  of  30  honeycomb  cells  that 
form  an  infinite  honeycomb  lattice  when  the  72- particle  configuration  is  used  to  tile  the  plane.  The 
value  of  the  Lindemann  parameter  is  calculated  using  the  initial  and  final  lattice  configurations. 
The  wall  clock  time  to  compute  the  Lindemann  parameter  on  a  single  CPU  is  approximately  2 
seconds. 

Optimizing  the  Lindemann  parameter  is,  however,  an  indirect  method  in  that  the  quanitity 
being  optimized  is  not  the  quantity  that  will  be  used  to  determine  the  final  quality  of  the  poten¬ 
tial.  A  more  direct  approach  is  to  explicitly  optimize  the  lattice  quality  metrics.  Although  the 
quality  metrics  require  a  slow  cooling  simulation  and  are  consequently  more  expensive  to  evaluate, 
optimization  of  the  quality  metrics  guarantees  optimization  of  lattice  quality.  Indeed,  an  important 
observation  made  in  this  paper  is  that  optimizing  the  Lindemann  parameter  is  only  moderately 
correlated  with  lattice  quality  -  potentials*  can  be  found  that  produce  a  low  value  of  the  Lindemann 
parameter,  yet  when  tested  in  a  slow  cooling  simulation  produce  lattices  of  poor  quality. 
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B.  Quality  Metric  Objective  Functions 

We  also  consider  direct  evaluation  of  the  Template  Measure  and  Defect  Measure  quality  metrics 
as  objective  functions.  To  evaluate  these  objective  functions,  we  start  with  a  regular  Cartesian  grid 
of  64  particles  with  a  spacing  that  provides  a  particle  density  equal  to  the  density  of  the  honeycomb 
lattice.  These  particles  are  initialized  with  a  temperature  well  above  the  lattice  melting  point 
and  then  this  temperature  is  slowly  reduced  until  the  particles  freeze  into  a  lattice  configuration. 
The  quality  metrics  are  computed  using  only  the  final  lattice  configuration.  Compared  with  the 
computation  of  the  Lindemann  parameter,  these  cooling  simulations  are  expensive  to  carry  out;  a 
single  call  to  a  quality  metric  objective  function  on  a  single  CPU  takes  approximately  70  seconds. 

It  should  be  noted  that  the  computational  expense  associated  with  evaluation  of  the  objective 
functions*  arises  almost  entirely  from  the  molecular  dynamics  simulations.  Computation  of  the 
actual  value  of  the  Lindemann  parameter,  the  Template  measure,  or  the  Defect  measure  after  the 
final  configuration  has  been  obtained,  requires  less  than  a  tenth  of  a  second. 

C.  Properties  of  the  Objective  Functions 

The  objective  functions  presented  have  important  features  that  influence  the  effectiveness  of  the 
optimization  schemes  employed.  One  of  the  chief  contributions  of  this  paper  is  the  use  of  a  trend 
optimization  scheme  that  is  better  suited  to  these  objective  functions  over  a  standard  simulated 
annealing  optimization  procedure. 

The  fact  that  the  objective  function.s  require  evaluation  times  on  the  order  of  seconds  and 
minutes  implies  that  any  optimization  method  that  requires  many  thousands  of  evaluations  to 
search  the  four-dimensional  parameter  space  will  necessitate  a  computation  time  measured  in 
hours  and  days.  The  time  taken  to  run  an  optimization  algorithm  will  be  spent  almost  entirely 
evaluating  the  objective  functions,  and  overhead  computation  required  by  the  optimization  scheme 
in  choosing  the  next  location  at  which  to  evaluate  the  objective,  for  example,  is  practically  negligible 
in  comparison.  The  trend  optimization  method  we  propose  is  particularly  well-suited  for  this 
situation  in  which  the  objective  functions  are  expensive  to  evaluate. 

Figure  6  displays  repeated  evaluations  of  the  Lindemann  parameter  objective  function  as  each 
of  the  parameters  in  V^hc(^;<2o,<2i»U2i03)  is  varied  in  turn,  while  the  remaining  parameters  are 
held  fixed  at  the  values  provided  by  Rechstman  et  al  (namely,  ao=5.89,  ai  =  17.9,  02=2.49,  and 
03=1.823).  Each  circle  in  the  plots  corresponds  to  a  single  evaluation  of  the  the  Lindemann 
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parameter  objective  function.  The  red  line  in  each  plot  represents  a  trend  line  that  is  computed 
by  averaging  over  100  samples  taken  at  each  of  600  regularly  spaced  intervals  along  the  axis  of  the 
varied  parameter. 


FIG.  6:  Each  circle  in  the  figures  above  corresponds  to  a  sample  evaluation  of  the  the  Lindcmann  parameter  objective 
function.  The  parameter  shown  on  the  x-axis  is  varied  over  the  indicated  domain,  while  the  remaining  parameters  are 
held  fixed  at  the  values  provided  by  Rechstman  et  al  (ao=5.89,  ai  =  17.9,  02=2.49,  and  03=1.823).  The  vertical  axis 
in  each  plot  represents  the  value  of  the  Lindcmann  parameter.  Note  that  the  vertical  scale  in  each  case  is  different. 
The  red  line  is  computed  by  averaging  over  100  trials  at  each  location,  and  reveals  a  smooth  trend  in  the  data. 

Our  first  observation  is  that  due  to  randomness  induced  by  the  initial  conditions  and  simulation 
at  constant  temperature,  the  objective  function  does  not  produce  repeatable  values  for  a  fixed 
set  of  parameter  values.  Repeated  evaluation  of  the  objective  functions  for  the  same  parameter 
values  leads  to  a  wide  range  of  output  values.  For  this  reason,  the  objective  functions  are  not 
strictly  functions,  although  we  continue  to  use  this  term.  More  correctly,  due  to  the  randomness 
introduced  by  the  initial  conditions  and  the  simulation  with  a  thermostat,  we  can  think  of  the 
objective  functions  as  assigning  a  one  dimensional  probability  distribution  to  each  set  of  values 
that  define  the  potential.  Thus,  a  call  to  the  objective  function  returns  a  value  drawn  from  the 
probability  distribution  specified  by  the  parameters.  We  shall  see  that  the  trend  optimization 
method  exploits  the  fact  that  the  expectation  values  of  the  probability  distributions  vary  smoothly 
with  respect  to  changes  in  the  parameters  of  the  potential. 

Examining  the  evaluations  of  the  objective  functions  reveals  that  they  are  noisy  with  respect 
to  variations  in  the  parameters.  The  functions  are  not  smooth  and  certainly  no  derivatives  of  the 
objectives  are  available.  Nevertheless,  averaging  over  many  evaluations  for  a  fixed  set  of  parameter 
values  reveals  that  the  objective  functions  do  possess  a  smooth  and  slowly  varying  trend.  As 
indicated  in  Figure  6,  the  smoothness  of  the  average  values  (shown  in  red)  indicate  that,  when 
averaged,  the  objective  functions  do  admit  a  sensible  notion  of  a  minimum. 
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In  summary,  the  salient  features  of  the  objective  functions  in  the  self-assembly  problem  are  the 


following: 

1 .  They  are  expensive  to  evaluate; 

2.  They  are  highly  variable — repeated  evaluations  of  the  objective  functions  for  the  same  input 
potential  yields  a  broad  range  of  values; 

3.  They  are  highly  non-smooth  with  respect  to  changes  in  the  parameters; 

4.  They  have  a  smooth  trend  when  averaged  over  many  evaluations. 

Objectives  with  these  features  are  often  encountered  in  optimal  design  problems  in  which  evalu¬ 
ation  of  the  objective  function  for  a  specific  set  of  parameter  values  requires  completion  of  an  actual 
laboratory  experiment,  or  the  execution  of  a  computationally  expensive  simulation  using  random 
initial  conditions  [28.  29].  Optimization  of  these  objective  functions  is  clearly  not  tractable  using 
standard  gradient-based  methods.  In  this  paper,  we  implement  an  optimization  scheme  ideally 
suited  for  objective  functions  of  this  type  that  has  allowed  us  to  construct  an  efficient  procedure 
for  solution  of  the  self-assembly  problem,  an  approach  that  we  call  trend  optimization. 

V.  OPTIMIZATION  METHODS 

In  the  previous  section,  various  objective  functions  have  been  introduced  that  can  be  used  in  the 
search  for  lattice- forming  potentials.  In  this  section,  we  present  two  methods  for  finding  optimal 
values  of  these  objectives.  F’irst,  we  briefly  discuss  simulated  annealing  which  was  used  by  [23]  in 
the  baseline  approach.  Second,  wc  present  in  detail  the  trend  optimization  approach. 

A.  Simulated  Annealing 

Simulated  annealing  mimics  the  ability  of  a  thermal  process  to  search  a  configuration  space  to 
find  the  ground  state.  We  begin  the  search  by  evaluating  the  objective  at  an  initial  point,  Iq?  in 
the  parameter  space  and  denote  the  value  of  the  objective  function  at  this  location  by  Eq.  The 
search  then  proceeds  by  jumping  to  new  locations  in  a  random  walk.  The  probability,  p,  of  making 
the  jump  from  Xi  to  a  new  location  Xj+i  is  determined  by 
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The  small  probability  of  moving  to  a  location  where  the  objective  is  in  fact  higher  than  the 
current  value,  allows  for  the  search  to  escape  from  a  local  minimum.  In  analogy  with  true  simulated 
annealing  in  metals,  the  temperature.  T,  is  slowly  reduced  so  that  at  first  the  search  eagerly 
traverses  the  search  space  by  easily  escaping  from  local  minima,  and  then  settles  to  a  specific 
local  minimum  as  the  temperature  decreases.  This  simulated  annealing  optimization  scheme  was 
implemented  using  the  GNU  Scientific  Library  GSL.SIMAN_SOLVE( )  routine  [30]. 

When  applied  to  the  self-assembly  problem,  the  simple  simulated  annealing  method  described 
here  fails  to  converge  to  a  sensible  minimum  because  of  noise  in  the  objective  functions.  Under¬ 
standably,  the  method  gets  stuck  in  ephemeral  local  minima  that  appear  and  disappear  due  to 
noise  in  the  objectives.  Convergence  can  be  obtained  if  the  objectives  are  sufficiently  smoothed. 
Smoothing  necessitates  averaging  over  at  least  20  independent  simulations  and  thus  requires  added 
computational  expense,  and  even  then  a  large  proportion  of  optimizations  fail  to  converge  w  ithin 
a  reasonable  number  of  objective  evaluations. 

Admittedly,  the  simulated  annealing  method  described  here  represents  a  very  simple  approach, 
and  more  complex  methods  involving  adaptive  search,  for  example,  could  be  employed.  The 
straightforward  simulated  annealing  approach  serves,  therefore,  as  a  modest  but  easily  understood 
baseline  method.  We  have  investigated  the  simulated  annealing  method  using  many  different 
cooling  regimens  including  adaptive  methods,  and  have  found  little  improvement  in  each  case. 

B.  Trend  Optimization 

Trend  optimization  is  well-suited  for  problems  in  which  the  objective  functions  are  expensive 
to  evaluate,  derivatives  of  the  objectives  are  not  available,  and  the  objectives  are  noisy  yet  exhibit 
a  simple  underlying  trend  when  averaged  over  many  evaluatioas. 

In  the  trend  approach,  we  use  a  few  w^ell-distributed  evaluations  of  the  objective  function  to 
generate  a  smooth  global  approximation  to  the  averaged  objective  function.  This  smooth  approx¬ 
imating  surface,  referred  to  as  the  surrogate^  attempts  to  capture  the  underlying  smooth  trend  in 
the  noisy  objective  evaluations.  The  optimization  then  proceeds  by  finding  the  optimal  values  of 
this  surrogate  function  that  is  computationally  cheap  to  evaluate.  Trends  in  the  surrogate  quickly 
reveal  regions  of  the  parameter  space  in  which  the  optimal  parameters  are  most  likely  to  be  found 
and  hence  the  search  is  greatly  accelerated.  The  insight  provided  by  the  smooth  surrogate  function 
then  informs  the  choice  of  parameters  at  w'hich  subsequent  evaluations  of  the  objective  should  be 
made. 
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Booker  et  al  [31]  combined  the  speed  and  facility  with  which  the  trend  approach  zooms  in  on 
regions  of  optimal  parameter  values  with  the  rigorous  convergence  guarantees  of  patterned  search 
methods  developed  previously  by  Torczon  [32],  to  produce  what  is  known  in  the  literature  as  the 
Surrogate  Management  Framework.  In  this  two- pronged  approach,  the  trend  method  is  used  in  the 
global  search  step  of  the  patterned  search  method  to  accelerate  the  search,  while  the  use  of  a  polling 
step  on  a  patterned  conceptual  grid  provides  the  guarantees  of  convergence.  In  this  seminal  paper, 
Booker  et  al  also  applied  the  Surrogate  Management  FVamework  approach  to  the  optimal  design 
of  a  helicopter  rotor  blade  with  thirty-one  design  variables.  More  recently,  Audet  et  al  [33,  34] 
have  provided  a  generalization  of  Torczon ’s  pattern  search  method,  which  they  refer  to  as  Mesh 
Adaptive  Direct  Search,  and  have  applied  it  to  optimization  of  the  chemical  treatment  of  discarded 
potliners  to  minimize  release  of  toxic  waste  in  the  production  of  aluminum  [35].  Mesh  Adaptive 
Direct  Search  has  since  been  incorporated  into  the  Surrogate  Management  Framework  by  Marsden 
et  al  in  [36]. 

The  range  of  design  problems  to  which  the  trend  optimization  approach  has  been  applied  is 
starting  to  grow.  Marsden  et  al  have  used  the  Surrogate  Management  Framework  in  the  optimal 
design  of  airfoils  to  reduce  noise  generated  in  the  trailing  turbulent  flow  [37],  and  a  computational 
framework  has  been  provided  in  [36]  for  optimizing  design  of  surgeries  for  improved  blood  flow 
and  cardiovascular  geometry.  Siah  et  al,  have  used  Kriging  surrogate  models  to  design  optimal 
configurations  and  shapes  of  automobile  antennae  to  minimize  electromagnetic  coupling  [38],  and 
Raza  et  al  have  compared  methods  for  generating  surrogate  functions  in  a  design  problem  seeking 
the  optimal  arrangement  of  fuel  rods  in  a  liquid  metal  reactor  [39]. 

A  unifying  theme  in  all  these  applications  is  the  parsimonious  way  in  which  trend  optimization 
is  able  to  optimize  expensive,  noisy  objective  functions.  Moreover,  trend  optimization  is  robust 
to  noise  in  that  the  trends  approximate  the  general  shape  of  the  objective  function  with  smooth 
surfaces  that  quieten  the  noise  and  anomalous  evaluations  of  the  objective  function.  Hence,  trend- 
based  approaches  survey  the  parameter  landscape  for  large  depressions,  and  are  not  distracted 
by  superficial  deep  spikes  that  may  arise  due  to  noise.  Because  the  surrogate  prioritizes  regions 
that  consistently  perform  well,  rather  than  a  single  “flash-in-the-pan”  evaluation,  the  parameter 
values  returned  by  the  trend  are  robust  to  uncertainties  and  are  more  likely  to  reliably  reproduce 
near-optimal  values  of  the  objective  upon  repeated  e\Tiluation. 

Trend  optimization  can  be  performed  in  a  coarse- to- fine  hierarchical  manner  by  recursively 
building  a  hierarchy  of  trend-fitting  surfaces.  Each  successive  iteration  of  the  procedure  yields  a 
new  trend  that  focuses  on  the  most  optimal  region  of  the  search  space.  Successive  trend  surfaces 
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Utilize  all  objective  function  evaluations  obtained  in  previous  iterations  to  more  accurately  model 
the  objective  function.  After  an  initial  global  trend  is  developed,  the  search  area  is  refined  to  the 
area  surrounding  the  global  minimum  of  the  surrogate  -  recall  that  since  the  surrogate  is  smooth 
and  cheap  to  evaluate,  the  global  minimum  of  the  surrogate  can  easily  be  found.  Refinement  of  the 
search  area  helps  to  ensure  that  subsequent  function  evaluations  are  chosen  in  locations  that  are 
most  relevant  and  promising.  As  the  search  area  becomes  more  refined,  successive  iterations  may 
use  a  larger  basis  of  fitting  functions,  or  use  more  sophisticated  trend  construction  methods  to  more 
accurately  pinpoint  the  location  of  the  minimum.  These  features  of  iterative  trend  optimization 
yield  a  hierarchy  of  coarse  to  fine  trends  that  enable  the  method  to  initially  make  large  strides 
toward  the  optimal  value,  and  then  to  focus  ever  more  tightly  on  the  exact  location  of  the  optimaJ 
value. 

In  our  discussion  thus  far,  it  remains  to  be  described  how  the  surrogate  functions  are  generated 
from  a  small  number  of  function  evaluations.  This  topic  lies  within  the  province  of  data  approxima¬ 
tion  and  fills  a  large  body  of  literature.  Needless  to  say,  there  are  a  great  number  of  interpolation 
and  fitting  approaches  available.  Popular  methods  include  polynomial  interpolation,  splines,  Krig> 
ing,  distance- based  interpolation,  linear  and  nonlinear  regressions,  radial  basis  functions,  neural 
networks,  and  kernel-based  approaches  [40-42].  For  a  thorough  survey,  please  see  the  monograph 
by  Hastie  [43]. 

Reviewing  the  Lindemann  parameter  objective  evaluations  depicted  in  Figure  6,  we  plainly  see 
that  an  interpolating  scheme  is  not  appropriate  for  the  noisy  objective  functions  of  the  self-assembly 
problem.  Pbr  this  reason,  we  have  chosen  the  ridge  regression  method  for  generating  surrogate 
functions  that  is  particularly  well-suited  for  noisy  data  in  high  dimensions.  Ridge  regression  was 
originally  developed  by  Tikhonov  (hence  the  method  is  sometimes  referred  to  as  Tikhonov  regular¬ 
ization)  for  ill-conditioned  linear  regression  problems  [44].  His  approach  was  to  introduce  diagonal 
stabilization  to  regularize  the  linear  interpolation  system.  The  added  regularization  improves  the 
conditioning,  and  introduces  smoothing. 

In  the  current  context,  we  are  provided  with  a  vector  of  M  noisy  measurements,  y  = 
[yii**'  ?yA/]?  of  the  objective  function  at  the  vector  of  locations  x  =  [xi,***  ,xa/].  We  want 
to  find  a  smooth  function  that  best  represents  the  smooth  trend  in  this  data.  This  trend,  denoted 
T(x).  is  constructed  as  the  weighted  sum  of  basis  functions: 

M 

T{x)  =  '^Ck^{x,Xk) , 

fc=l 

where  we  must  now  solve  for  the  vector  of  coefficients  c  =  [ci,-*-  ,ca/].  Gaussian  radial  basis 
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functions  are  chosen  for  the  basis  since  they  are  well-suited  for  regression  problems  in  high  dimen¬ 
sions  [45-47].  To  be  specific,  we  choose  radial  basis  functions  of  the  form 

^X,Xj)  =  oidlx  -  Ij||2) 

where 

,/,(r)  =  reR. 

Proceeding  in  the  standard  manner  for  linear  regression,  the  vector  of  coefficients,  c,  in  the  expres¬ 
sion  for  the  trend  are  obtained  by  solving  the  linear  system 

Ac  =  y  (6) 


where  the  elements  of  the  square  symmetric  matrix  A  are  given  by 


Aij  :=  . 


This  procedure  assumes  that  A  is  full  rank.  In  ridge  regression,  the  conditioning  of  A  is  improved 
by  adding  diagonal  regularization.  The  linear  system  in  (6)  above  is  replaced  by 


c  = 


y 


(7) 


in  which  cj  6  R,  and  I  is  the  identity  matrix.  Regularization  is  obtained  at  the  expense  of  introduc¬ 
ing  the  new'  free  parameter  w.  In  the  traditional  usage  of  ridge  regression,  the  analyst  must  choose 
an  optimal  value  of  u;  that  balances  the  need  for  improved  conditioning  with  the  desire  to  keep 
the  departure  from  the  least-squares  solution  small.  In  our  context,  since  we  are  not  immediately 
concerned  with  conditioning,  we  use  uj  to  control  the  amount  of  smoothing  introduced.  Smaller 
values  of  uj  yield  larger  smoothing,  while  larger  values  of  w  ensure  increased  pointwise  accuracy  to 
the  noisy  data.  In  practice,  this  approach  is  straightforward  and  robustly  produces  smooth  trends 
to  noisy  objective  functions.  In  Figure  7,  an  illustration  of  smooth  trends  generated  using  ridge 
regression  are  given  for  noisy  evaluations  of  the  Lindemann  parameter.  It  must  be  remembered 
though  that  these  surrogates  are  constructed  as  one-dimensional  curves  for  illustration,  whereas  in 
the  full  self-assembly  problem  the  surrogate  functions  are  smooth  four-dimensional  hypersurfaces. 

The  ridge  regression  method  can  be  extended  to  include  adaptive  control  of  the  amount  of 
smoothness  introduced  in  response  to  local  conditions.  This  local  ridge  regression  approach  is 
implemented  by  replacing  Equation  (7)  with 


^A  +  diag 


1 

2uii ' 


=  y 
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FIG.  7:  Generating  surrogate  functions  for  noisy  data.  Each  circle  represents  an  evaluation  of  the  Lindemann 
parameter  as  in  Figure  6.  The  solid  trend  line  is  generated  using  Gaussian  radial  basis  functions  and  ridge  regression. 

where  the  vector  of  M  free  parameters  can  be  chosen  independently  to  adjust  the 

amount  of  smoothing  local  to  each  measurement.  In  this  way,  a  low  value  of  u;  can  be  chosen  to 
locally  increase  the  smoothing  in  a  region  in  which  the  noise  in  the  data  is  high,  and  similarly,  a 
large  value  of  cj  can  be  chosen  for  measurements  in  regions  in  which  there  is  high  confidence  in 
the  data  and  noise  is  low.  This  facility  is  not  currently  implemented  in  our  trend  optimization 
algorithm,  but  is  mentioned  here  to  indicate  the  flexibility  that  the  ridge  regression  method  affords. 

As  previously  mentioned,  there  are  many  candidate  approaches  for  trend  fitting  besides  ridge 
regression  that  could  be  uses  in  the  self-assembly  problem.  For  instance,  simple  quadratic  fitting  is 
very  easily  implemented  and  guarantees  convexity  of  the  surrogate.  In  practice,  this  method  also 
works  remarkably  well  for  the  objectives  of  the  self-assembly  problem. 


Tivnd  Optimization  AlgoHthm  used  in  the  Self-Assembly  Problem 

In  using  the  trend  optimization  approach  for  the  design  of  the  potentials  in  the  self-assembly 
problem,  we  have  implemented  it  without  coupling  to  a  Direct  Search  method.  Consequently, 
we  lose  rigorous  guarantees  of  convergence,  but  in  practice  trend  optimization  alone  performs 
remarkably  well  and  reliably  converges  to  an  optimal  value,  as  will  be  demonstrated  over  repeated 
trials. 

Throughout  the  computations,  we  scale  the  four-dimensional  parameter  search  space  to  form 
a  unit  hypercube.  VVe  use  the  hierarchical  approach  described  above  with  three  levels  of  recur¬ 
sion.  Hence,  during  each  optimization  three  trend  surfaces  will  be  constructed  at  finer  and  finer 
resolution.  The  basic  steps  of  the  procedure  are  as  follows. 

Let  U\  denote  the  four-dimensional  unit  hypercube  of  the  parameter  space  to  be  searched,  and 
let  H  be  the  number  of  levels  in  the  trend  hierarchy  (we  use  H  =  3). 
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FOR  each  iteration  in  the  hierarchical  trend  optimization  method,  indexed  by  A:  =  •  •  •  ,  //: 

Step  1 .  Generate  Sample  Locations: 

Select  A/  points,  [zp---  from  Uk  u.sing  Latin  Hypercube  Sampling  [48]. 

Step  2.  Evaluate  the  Objective: 

Evaluate  the  objective  at  each  of  the  A/  locations  and  store  each  corresponding 
result  in  y^. 

Step  3.  Build  the  Trend: 

Use  all  data  :j  =  l,--,A:;i  =  l,---,A/|  obtmned  during  the  optimization 

so  far  to  construct  the  trend  surface  Tjt  via  ridge  regression. 

Step  4.  Optimize  the  Trend: 

Quickly  find  xj,  the  location  in  the  parameter  space  that  globally  minimizes  the  trend 
surface  T^. 

Step  5.  Refine  the  Search  Domain: 

Generate  a  new  search  domain,  Uk+ii  by  reducing  the  size  of  the  current  search  domain, 
by  a  factor  of  2  along  each  dimension  centered  about  the  |X)int  xj. 


END 

After  H  iterations,  declare  x*  :=  as  the  parameter  location  that  minimizes  the  objective 
function.  If  desired,  the  objective  function  can  be  evaluated  repeatedly  at  x*,  and  then  averaged, 
to  generate  y*.  the  expected  value  of  the  objective  at  x*. 

The  most  computationally  expensive  step  is  Step  2,  the  evaluation  of  the  objective  function. 
In  comparison,  optimization  of  the  surrogate  performed  in  Step  4  is  extremely  fast.  To  effect 
Step  4,  we  simply  evaluated  the  surrogate  at  5000  points  randomly  distributed  throughout  the 
search  s'pace  and  selected  the  point  that  produces  the  lowest  value  of  the  surrogate.  The  surrogate 
provides  repeatable  values  so  there  is  no  need  to  average  over  many  evaluations  of  the  surrogate. 

At  the  completion  of  the  algorithm,  the  total  number  of  required  objective  function  evaluations 
is  A/  ■  //,  that  is,  A/  objective  evaluations  during  each  of  the  //  trend  iterations.  An  important 
point  is  that,  at  each  iteration,  the  A/  objective  evaluations  required  by  Step  2  are  independent^ 
meaning,  therefore,  that  these  A/  evaluations  can  be  computed  in  parallel.  In  our  optimization 
source  code,  we  have  parallelized  Step  2  so  that  the  total  wall  clock  time  required  for  the  entire 
optimization  algorithm  to  complete  is  H  •  Tobj,  where  Tobj  is  the  CPU  time  required  to  run  the 
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optimization  with  a  single  objective  function  evaluation.  An  important  point,  therefore,  is  that  if 
the  number  of  available  computing  processors  is  greater  than  M  (and  we  consider  any  overhead 
communication  costs  between  the  M  processors  as  negligible),  then  the  execution  time  of  the 
algorithm  is  independent  of  Af.  Thus,  trend  optimization  provides  an  effective  method  to  harness- 
parallel  computation  resources  for  fast  global  optimization. 

Since  in  our  specific  implementation  of  the  trend  algorithm  we  recursively  generate  three  sur¬ 
rogate  functions,  and  since  the  evaluation  of  even  the  most  expensive  objective  function  is  approx¬ 
imately  a  minute;  the  trend  algorithm  terminates  after  Just  three  minutes  of  computation  on  40 
processors  having  used  information  gathered  from  120  objective  evaluations.  The  CITerra  cluster 
at  Caltech  has  4096  processes  that  could  conceivably  be  used  to  find  an  optimal  solution;  however, 
we  have  observed  that  trend  optimization  provides  reliable  convergence  when  M  (the  number  of 
objective  evaluations  made  at  each  step)  is  ~  40  or  above,  so  that  a  far  more  modest  number  of 
processors  is  actually  required.  In  summary,  the  trend  optimization  scheme  as  we  have  applied 
it  to  the  self-assembly  problem  is  able  to  provide  parameters  that  optimize  the  most  expensive 
objective  functions  in  Just  over  three  minutes  when  run  on  a  cluster  of  40  parallelized  processors. 

It  should  be  noted  that  the  hundredfold  speed-up  obtained  by  trend  optimization  over  simulated 
annealing  in  the  time  taken  to  generate  potentials  (as  stated  in  the  introduction),  is  assessed 
using  the  total  CPU  time  summed  over  all  processors,  and  not  the  wall  clock  time.  The  speed¬ 
up  is  attributed  to  the  robust  manner  in  which  trend  optimization  accelerates  search  of  a  noisy 
objective  with  a  smooth  trend.  Hence,  the  speed-up  attributed  to  the  facility  with  which  the  trend 
optimization  admits  parallelization  of  the  computation  represents  an  additional  time  savings  over 
and  above  the  hundredfold  speed-up. 

VI.  METHODS  FOR  GENERATING  POTENTIALS 

The  five  methods  for  generating  potentieds  that  we  compare  in  this  paper  are  described  below. 
The  first  is  a  heuristic  geometric  method  that  requires  no  computation.  The  other  four  methods 
utilize  an  optimization  procedure. 

A.  Geometric  Method 

The  geometric  method  (abbreviated  as  GM)  that  we  present  here  is  an  optimization-free  pro¬ 
cedure  that  exploits  differences  between  the  geometry  of  the  desired  target  lattice  and  competitor 
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lattices  that  we  hope  to  discourage.  The  design  of  the  potential  is  based  on  four  main  principles: 

GMl.  The  potential  must  have  local  minima  located  at  each  of  the  radial  distances  to  nearest 
neighbors  in  the  desired  lattice  and  nowhere  else; 

GM2,  The  potential  may  include  local  maxima  at  radial  distances  at  which  competitor  lattices 
have  nearest  neighbors  but  the  target  lattice  does  not; 

GM3.  When  the  target  and  competitor  lattices  have  nearest  neighbors  at  the  same  radiaJ  dis¬ 
tances,  the  energy  levels  of  the  local  minima  identified  in  GMl  are  chosen  to  energetically 
prefer  the  target  lattice; 

GM4.  A  Lennard- Jones  potential  is  spliced  into  the  potential  at  the  origin  to  provide  a  hard 
core  potential. 

GMl  and  GM2  use  information  about  the  geometric  structure  of  the  target  and  competitor 
lattices  to  stabilize  the  target  lattice,  and  to  discourage  a  competitor  lattice  that  has  different 
distances  to  nearest  neighbors.  GM3  uses  energetics  to  discriminate  between  lattices  that  have 
identical  distances  to  nearest  neighbors  and  differ  only  in  the  numbers  of  particles  at  those  distances. 

When  the  target  lattice  is  the  square  lattice,  and  the  identified  competitor  is  a  triangular 
lattice,  the  fact  that  these  lattices  have  different  distances  to  nearest  neighbors  makes  this  method 
of  potential  generation  a  very  robust  approach.  By  explicit  construction,  the  locations  of  the  local 
minima  and  maxima  discourage  the  triangular  lattice  and  stabilize  the  square  lattice.  The  shape  of 
the  potential  generated  for  this  case  is  shown  in  Figure  8(a).  Notice  in  particular  that  the  potential 
has  a  very  simple  form  with  local  minima  at  distances  of  1.0  and  \/2  to  encourage  the  square  lattice, 
and  a  local  maximum  located  at  \/3  to  discourage  the  triangular  lattice.  In  simulation  this  potential 
performs  remarkably  well.  The  potential  robustly  forms  the  square  lattice,  and  defects  (except  for 
voids)  are  seldom  observed.  A  sample  square  lattice  obtained  with  this  potential  is  shown  in 
Figure  8(b). 

The  honeycomb  lattice  represents  a  more  difficult  self-assembly  problem  since  both  this  target 
lattice  and  the  competitor  triangular  lattice  have  identical  distances  to  nearest  neighbors.  The 
only  difference  between  these  lattices  that  must  be  exploited  is  the  different  number  of  neighbors 
at  each  distance.  Hence,  in  GM3.  the  relative  heights  of  the  local  minima  are  chosen  to  ensure 
that  the  target  lattice  represents  a  lower  energy  state  than  the  competitor  lattice  and  is  thus  more 
likely  to  form  upon  cooling. 
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FIG,  8:  Potentials  developed  using  the  heuristic  geometric  method  for  both  the  square  and  honeyeomb  lattiees  are 
shown  in  Figure  (a).  Figures  (b)  and  (e)  show  typieal  lattices  obtained  using  these  potentials. 


The  application  of  the  GM  method  for  the  specific  case  of  constructing  a  potential  that  favors 
the  honeycomb  lattice  and  discourages  the  triangular  lattice  proceeds  as  follows: 

1.  Recognize  that  both  the  triangular  and  honeycomb  lattices  have  nearest  neighbors  at  a 
distance  of  1.0,  and  second  nearest  neighbors  at  a  distance  of  \/3.  The  number  of  neighbors 
at  those  distances  (the  lattice  coordination  numbers)  for  the  honeycomb  lattice  are  3  and  6. 
while  the  triangular  lattice  has  6  and  6. 

2.  Assign  local  minima  at  distances  of  1,0  and  \/3. 

3.  Since  the  triangular  lattice  has  more  particles  at  distance  1.0  than  the  target  lattice,  raise 
the  first  minimum  at  1.0  with  respect  to  the  height  of  the  minimum  at  v^. 

4.  Splice  in  a  6-12  Lennard- Jones  type  repulsive  potential  at  the  origin  to  simulate  a  rigid  core. 

5.  Use  cubic  splines  to  piece  together  the  repulsive  core  at  the  origin  and  the  local  minima  at 
their  various  locations  and  heights. 

One  other  consideration  that  we  include  when  designing  this  potential  is  that  we  are  careful 
not  to  make  the  local  maximum  between  the  first  and  second  minima  too  high.  This  ensures  that 
particles  that  are  stuck  in  the  triangular  formation  have  increased  likelihood  of  escaping  to  the 
lower  potential  well  of  the  honeycomb  lattice. 

The  cubic  splines  are  constructed  so  as  to  have  zero  derivative  at  the  local  minima,  thus  ensuring 
that  the  potential  is  continuously  differentiable  on  the  whole  positive  real  line.  A  sample  potential 
constructed  using  this  method  for  the  generation  of  honeycomb  lattice  potentials  is  shown  in 
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Figure  8(a).  Notice  that  the  potential  has  local  minima  located  at  distances  of  1,0  and  \/3,  with  the 
first  minimum  located  above  the  second  minimum.  Since  the  triangular  lattice  has  more  particles 
located  at  distance  1.0  than  the  honeycomb  lattice,  this  ensures  that  the  triangular  lattice  is  a 
higher  energy  state  than  the  honeycomb  lattice.  A  sample  lattice  produced  using  this  potential  is 
shown  in  Figure  8(c).  The  fact  that  this  heuristic  geometric  method  is  able  to  produce  honeycomb 
lattices  of  this  quality  without  having  to  use  any  computation  and  optimization  is  quite  remarkable 
and  until  now  has  been  overlooked  in  the  self-assembly  literature. 

VV'e  refer  to  the  geometric  method  as  a  heuristic  method  since  we  do  not  provide  formulas  or 
algorithms  for  choosing  the  relative  heights  of  the  local  minima  and  maxima.  The  only  prescribed 
constraints  are  the  locations  of  the  local  minima  that,  by  construction,  ensure  stability  of  the  target 
lattice.  In  practice,  we  find  that  any  sensible  choice  of  the  heights  that  avoids  sharp  gradients  in 
the  potential  works  reasonably  well.  In  the  implementation  of  our  computer  code,  the  user  need 
only  input  the  location  and  heights  of  the  desired  minima  and  maxima,  and  the  spline  fitting  and 
splicing  of  the  Lennard-Jones  potential  are  computed  automatically,  which  makes  the  approach 
very  simple  to  execute.  Given  the  simplicity  of  the  geometric  method,  the  ease  with  which  it 
is  implemented,  and  the  relatively  high  quality  of  the  resulting  lattices,  it  represents  a  “rough- 
and-ready”  approach  that  anyone  needing  to  design  a  potential  would  do  well  to  consider  before 
pursuing  more  computationally  exj>ensive  schemes. 

Whereas  the  geometric  method  utilizes  only  the  geometries  of  the  static  target  lattice  and 
competitor  lattice,  the  optimization  methods  discussed  next  incorporate  information  about  particle 
dynamics  by  optimizing  parameters  in  the  potential  with  respect  to  dynamic  particle  simulations. 

B.  Baseline  Method 

The  baseline  simulated  annealing  method  of  [23]  uses  the  simulated  annealing  optimization 
procedure  with  the  Lindemann  parameter  as  the  objective  function.  As  mentioned  previously,  the 
simulated  annealing  procedure  fails  to  converge  to  a  meaningful  minimum  unless  the  objective 
function  is  averaged  over  many  trials.  We  refer  to  this  method  with  the  abbreviation  SA-LP20  to 
indicate  that  the  simulated  annealing  approach  is  applied  to  the  Lindemann  parameter  averaged 
over  20  independent  evaluations.  The  large  number  of  samples  required  to  sufficiently  reduce 
the  noise  in  the  objective  function,  means  that  it  is  impractical  to  apply  the  simulated  annealing 
optimization  procedure  to  the  more  expensive  quality  metric  objective  functions.  The  simulated 
annealing  procedure  is  initiated  with  initial  guess  solutions  chosen  randomly  and  uniformly  from 
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Suimiiary  of  Potential  Generation  Methods 

Method 

Optimization 

Objective 

Cost  (s) 

GM 

Geometric  method 

0 

5A-LP20 

Sim.  anneal. 

Lindeniann  (20  sample  avg.) 

40 

T-LP 

TVend 

Lindeniami 

2 

T-TM 

TYeiid 

Template  Measure 

70 

T-DM 

TYend 

Defect  Measure 

70 

TABLE  I:  Each  of  the  five  methods  for  generating  potentials  is  listed  here  with  the  associated  computational  cost 
of  evaluating  the  objective  function.  Notice  that  the  Geometric  Method  is  not  an  optimization- based  method  and 
incurs  no  computational  cost. 

the  parameter  space  described  by  the  inequalities  in  line  (2)  of  Section  II. 

C»  Trend  Optimization  Methods 

The  remaining  three  methods  that  we  consider  utilize  the  trend  optimization  method  applied  to 
the  Lindemann  parameter,  the  Template  Measure,  and  the  Defect  Measure  as  objective  functions. 
These  methods  are  abbreviated  T-LP,  T-TM.  and  T-DM  respectively.  Since  the  trend  optimization 
method  is  well-suited  to  noisy  objective  functions,  the  objectives  do  not  need  to  be  averaged  over 
many  runs  when  evaluated. 

A  summary  of  all  the  potential  generation  methods  under  study  is  provided  in  Tabic  I. 

VII.  RESULTS 

In  this  section,  we  compare  the  effectiveness  of  the  proposed  methods  for  generating  interaction 
potentials  that  lead  to  the  self-assembly  of  the  honeycomb  lattice.  In  particular,  in  accordance 
with  the  comparison  criteria  enumerated  in  Section  II,  we  seek  answers  to  the  following  questions: 

1.  How  much  computational  effort  is  required  to  generate  the  potentials? 

2.  What  is  the  quality  of  the  lattices  generated  by  the  potentials? 

3.  How  reliably  do  the  potentials  form  high  quality  lattices  given  uncertainty  in  the  initial 
conditions  of  the  particles? 

Answers  to  these  questions,  as  well  as  a  comparison  of  the  five  potential  generation  methods 
discussed  in  Section  VI,  are  summarized  in  the  suite  of  plots  shown  in  Figures  9  and  10.  In 
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brief.  Figure  9  shows  the  superior  effectiveness  of  trend  optimization  over  simulated  annealing  in 
minimizing  the  respective  objectives  as  a  function  of  the  number  of  objective  evaluations  required, 
while  Figure  10  shows  the  quality  of  the  lattices  obtained  by  the  potentials  that  were  generated  by 
the  optimizations. 


(a)Simulatcd  annealing  optimization  of  (b)TVend  optimization  of  the 

the  Lindemann  parameter  (SA-LP20).  Lindemann  fjarameter  (T-LP). 


(e)Tyend  optimization  of  the  Template 
Measure  (T-TM). 


(diTYend  optimization  of  the  Defect 
Measure  (T-DM). 


FIC.  9:  Averaged  objective  funetion  values  versus  number  of  objeetive  function  evaluations  for  four  optimization 
methods  are  shown  here.  In  the  results  for  simulated  annealing  shown  in  (a),  dashed  lines  indieate  optimizations 
that  failed  to  find  an  optimal  value.  The  single  bold  line  represents  the  mean  of  the  optimizations  that  did  eonverge 
to  an  optimal  value  (indieated  by  solid  lines).  Most  notably,  trend  optimization  reliably  converges  to  a  minimum 
value  of  the  Lindemann  parameter  with  a  one-hundredfold  reduction  in  the  number  of  required  objective  function 
evaluations  when  compared  with  the  simulated  annealing  approach. 


The  geometric  method  (GM)  is  the  simplest  method  to  implement.  It  does  not  require  any 
objective  optimization  to  search  for  parameters  and  consequently  does  not  incur  any  computational 
cost.  In  order  to  evaluate  the  quality  of  the  lattices  produced  by  the  geometric  method,  one-hundred 
cooling  simulations  were  performed  using  the  honeycomb  potential  shown  in  Figure  8(a).  After 
each  simulation,  the  Template  Meas*ure  and  the  Defect  Measure  were  computed.  Averaging  over 
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all  these  runs  yielded  the  following  scores: 

1.  Template  Measure  =  17,9  with  standard  deviation  3.4, 

2.  Defect  measure  =11.8  with  standard  deviation  4.5. 

Comparison  with  the  lattice  qualities  obtained  using  the  optimization-based  methods  as  shown  in 
Figure  10  indicates  that  these  quality  scores  are  exceptionally  good,  and  that  assiduous  computa¬ 
tional  optimization  is  required  to  produce  potentials  that  produce  lattices  of  higher  quality. 

The  four  optimization-based  methods  (SA-LP20,  T-LP,  T-TM,  and  T-DM)  were  each  run  for  an 
increasing  number  of  allowed  objective  function  evaluations.  For  each  number  of  objective  func¬ 
tion  evaluations,  each  trend  optimization  method  w^as  executed  in  twenty  independent  trials,  w4th 
each  trial  generating  parameters  for  the  interaction  potential  that  seek  to  minimize  the  respective 
objective  function.  In  Figure  9,  the  average  of  the  values  of  the  objective  function  obtained  in 
the  twenty  trials  is  plotted  for  each  number  of  function  evaluations  (the  error  bars  indicate  the 
standard  deviation  over  the  twenty  trials  for  each  number  of  function  evaluations). 

After  all  the  optimizations  are  completed  and  the  potentials  have  been  generated,  the  quality 
of  each  potential  must  be  tested.  For  each  number  of  function  evaluations,  and  for  each  of  the 
twenty  independent  trials,  the  potential  produced  by  a  method  was  quality  tested  by  running 
twenty  cooling  simulations  on  a  system  of  225  particles,  and  then  measuring  the  quality  of  the  final 
lattices  using  both  the  Template  Measure  and  the  Defect  Measure.  For  a  single  trend  optimization 
method,  this  requires 

(12  different  numbers  of  function  evaluations \  /  20  independent  trials  N  /  20  cooling  simulations  \ 

each  trend  method  /  V  number  of  function  evaluations/  \  each  independent  trial  / 

(  4800  cooling  simulations  \ 
each  trend  method  / 

In  Other  words,  240  independent  optimization  trials  are  required  to  produce  a  single  curve  in  Figure 
9,  and  4800  cooling  simulations  are  required  to  produce  a  single  curve  in  Figure  10.  The  cooling 
simulations  were  initialized  with  a  temperature  approximately  1.5  times  the  melting  temperature 
of  the  lattice  and  then  slowdy  cooled  using  a  Nos^Hoover  thermostat  to  less  than  ten  percent  of  the 
melting  temperature.  At  the  completion  of  each  cooling  simulation,  both  the  Template  Measure 
and  the  Defect  Measure  were  computed  to  measure  the  quality  of  the  resulting  lattice.  Averaging 
over  the  20  cooling  simulations  yields  two  quality  scores  for  each  potential  -  one  for  each  quality 
metric.  Results  for  the  Template  Measure  arc  shown  in  Figure  10(c)  while  results  for  the  Defect 
Measure  are  shown  in  Figure  10(d). 
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After  reviewing  the  results  in  Figure  9,  we  see  that  the  trend  approach  clearly  provides  a  faster 
and  more  robust  method  for  optimizing  the  objective  functions.  Figure  9(a)  shows  the  results 
from  twenty  simulated  annealing  optimizations  for  different  randomly  chosen  stairting  points  in 
the  search  domain,  and  indicates  that  the  success  of  the  method  is  highly  variable.  Simulated 
annealing  often  fails  to  find  optimal  parameter  values  and  remains  stuck  in  local  minima  even 
after  10,000  evaluations  of  the  objective.  Of  the  twenty  simulated  annealing  optimization  trials 
performed,  only  nine  were  able  to  find  an  optimal  value  of  the  Lindemann  parameter  less  than 
0.15.  The  bold  line  in  Figure  9(a)  represents  the  mean  over  only  these  nine  best-performing 
trials.  Moreover,  many  of  these  simulated  annealing  optimizations  that  do  converge  require  more 
than  6,000  objective  evaluations.  In  contrast,  we  see  from  Figure  9(b)  that  trend  optimization 
reliably  finds  optimal  values  after  sixty  evaluations  of  the  Lindemann  parameter.  We  conclude 
that  when  trend  optimization  is  used  to  optimize  the  Lindemann  parameter,  we  obtain  a  one- 
hundred-fold  reduction  in  computation  time  over  the  simulated  annealing  method  of  [23],  and  that 
the  optimization  is  more  robust.  The  speed-up  can  be  attributed  to  the  fact  that  the  objective 
function  is  noisy  yet  has  a  simple  trend  two  properties  for  which  trend  optimization  is  ideally 
suited. 

Furthermore,  the  accelerated  search  of  the  trend  method  makes  it  possible  to  use  trend  opti¬ 
mization  with  the  more  expensive  quality  metrics  as  objective  functions.  Doing  so  provides  the 
extra  guarantee  that  the  generated  potentials  produce  high-quality  lattices.  As  indicated  in  Fig¬ 
ures  9(e)  and  9(d),  trend  optimization  reliably  finds  optimal  values  of  the  lattice  quality  objectives 
in  less  than  120  function  evaluations,  although  it  must  be  remembered  that  these  objectives  are 
approximately  35  times  more  expensive  to  evaluate  than  the  Lindemann  parameter.  Nevertheless, 
the  total  CPU  time  required  to  perform  these  optimizations  is  still  less  than  the  time  taken  by 
simulated  annealing  to  optimize  the  Lindemann  parameter. 

Figure  10  shows  the  quality  of  the  lattices  generated  by  the  potentials  produced  by  the  various 
methods,  as  measured  using  both  quality  metrics.  Several  important  observations  are  to  be  made 
after  reviewing  these  plots.  First,  optimization  of  the  Lindemann  parameter  leads  to  lattices  of 
modest  and  unreliable  quality,  indicating  that  the  Lindemann  parameter  and  lattice  quality  are 
only  moderately  correlated.  Second,  even  when  simulated  annealing  is  successful  in  optimizing  the 
Lindemann  parameter,  the  corresponding  potential  may  yield  lattices  of  poor  quality.  This  occurs 
because  simulated  annealing  may  find  minima  corresponding  to  very  narrow  wells  that  do  not  pro¬ 
vide  robustness  against  uncertainty  in  initial  conditions.  In  contrast,  since  trend  optimization  seeks 
out  the  general  trend  over  the  entire  search  space,  this  method  finds  minima  that  more  robustly 
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(a)Quality  of  lattices  generated  using 
simulated  annealing  measured  with  the 
Template  Measure. 


(c)Comparison  of  lattice  quality  using  the 
Template  Measure. 


Number  0<  FurKtton  Evatuetlone 


(b)Quality  of  lattices  generated  using 
simulated  annealing  measured  with  the 
Defect  Measure. 


(d) Comparison  of  lattice  quality  using  the 
Defect  Measure. 


FIG.  10:  Here  we  compare  the  quality  of  the  lattices  produced  using  the  four  optimization  methods.  In  (a)  and 
(b),  the  quality  of  the  lattices  produced  using  the  simulated  optimization  method  is  shown.  It  should  be  noted  that 
only  the  nine  potentials  generated  by  the  convergent  simulated  annealing  trials  in  Figure  9(a)  were  used  to  produce 
these  plots.  Figures  (c)  and  (d)  show  the  quality  of  lattices  produced  using  the  three  trend  optimization  methods. 
Using  trend  optimization  directly  on  the  quality  metrics  reliably  produces  high  quality  lattices.  Optimization  of  the 
Lindemann  parameter  is  only  moderately  correlated  with  improved  lattice  quality.  In  these  Figures,  recall  that  values 
of  the  Template  Measure  and  the  Defect  Measure  for  purely  random  configurations  arc  132  and  161,  respectively, 
and  that  the  computation-free  Geometric  Method  produces  lattice  quality  scores  of  18  and  12,  respectively. 


lead  to  high  quality  lattices.  Finally,  the  two  lattice  quality  metrics  are  reasonably  consistent  in 
that  a  potential  generated  by  optimizing  one  of  the  quality  measures  is  also  considered  high-quality 
in  the  other  measure. 

Figure  11  shows  the  range  of  potentials  generated  by  the  trend  optimization  methods.  The 
potential  provided  by  [23]  is  also  shown  for  reference.  Most  striking  is  that  trend  optimization  un- 
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FIG.  11:  Potentials  for  the  generation  of  honeycomb  lattices.  V/?  is  the  potential  previously  provided  by  [23].  TYcnd 
optimization  generates  potentials  similar  to  [23]  (labelled  VU),  but  also  uncovers  a  new  family  of  solutions  that  have 
a  more  repulsive  shape  and  no  local  minima  (sample  potentials  in  this  family  arc  labelled  Vb  and  V^c)  Remarkably, 
these  repulsive  potentials  robustly  form  large  regions  of  honeycomb  lattice  without  defects. 

covers  an  entirely  new  family  of  potentials  not  previously  considered.  In  this  family  of  potentials, 
the  exponential  term  is  dominant  and  consequently,  their  shape  does  not  admit  a  local  minimum. 
The  repulsive  shape  of  these  potentials  leads  to  higher  quality  lattices  since  particles  do  not  get 
stuck  in  local  minima  associated  with  local  potential  wells.  In  simulation,  we  observe  that  the  de- 
fcicts  continue  to  move  through  the  configuration  until  they  leave  the  domain  entirely,  or  annihilate 
one  another  through  collisions. 

Figure  12  shows  two  sample  lattice  configurations  obtained  for  4128  particles  simulated  using 
the  potential  labelled  Vc  in  Figure  11  that  was  generated  with  trend  optimization  on  the  Tem¬ 
plate  Measure.  The  parameter  values  for  this  potential  are  ao=5.771,  ai  =23.594,  02=0.574,  and 
03=1.816.  The  final  lattices  exhibit  large  regions  of  extremely  well-formed  honeycomb  lattice  that 
we  have  not  previously  observed  using  potentials  that  contain  local  minima.  In  Figure  12(a),  a 
prominent  grain  boundary  lies  between  two  large  areas  of  well-formed  honeycomb  lattice.  As  in 
Nature,  this  grain  boundary  forms  when  the  cooling  is  not  sufficiently  slow.  In  Figure  12(b),  iso¬ 
lated  defects  are  visible  in  the  lattice;  however,  it  should  be  noted  that  these  defects  are  not  yet 
“frozen’’  into  the  lattice,  and  given  enough  time  will  eventually  leave  the  domain  or  disappear 
through  mutual  collision. 

The  repulsive  potentials  found  using  trend  optimization  are  remarkably  effective  at  producing 
large  regions  of  almost  defect  free  honeycomb  lattice  despite  their  relatively  simple  shape.  The 
effectiveness  of  these  potentials  suggests  that  a  far  simpler  basis  of  potential  functions  can  be 
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(a)  (b) 


FIG.  12:  Honeycomb  lattices  formed  using  the  repulsive  family  of  potentials  discovered  using  trend  optimization, 
(a)  Two  large  regions  of  extremely  well- formed  honeycomb  lattice  meet  to  form  a  grain  boundary,  (b)  Large  regions 
of  the  honeycomb  lattice  are  formed  with  a  few  isolated  defects.  These  defects  continue  to  move,  even  after  the 
lattice  is  frozen,  until  they  exit  the  boundary  or  are  annihilated  by  collisions  with  other  defects. 


used  in  the  parameterization  of  VHc(r).  In  these  potentials  it  is  the  exponential  decay  term  that 
dominates  over  the  Gaussian  term.  The  numerical  simulations  seem  to  suggest,  and  it  would  be 
interesting  to  pursue  rigorously,  that  for  a  fixed  density,  the  honeycomb  lattice  is  a  global  minimizer 
(or  ground  state)  of  the  exponential  decay  pairwise  interaction  potential.  This  path  is  consistent 
with  the  numerical  findings  of  Jagla  et  al  in  [50]  for  a  ramp  type  potential. 

VIII.  ANISOTROPIC  POTENTIALS 

The  methods  presented  thus  far  can  be  used  to  quickly  generate  isotropic  potentials  that  produce 
high  quality  lattices.  However,  the  potentials  for  self-assembling  honeycomb  lattices  are  not  robust 
to  variations  in  density  -  the  lattices  only  form  if  the  initial  density  of  the  particles  is  very  near  the 
ideal  density  of  the  target  lattice.  If  the  initial  density  is  much  less  than  that  of  the  target  lattice, 
then  a  triangular  lattice  will  be  favored.  [23]  addressed  this  issue  by  searching  for  optimal  potentials 
over  a  range  of  densities.  However,  the  tolerance  for  variation  in  density  is  still  very  narrow  and 
may  indeed  be  a  fundamental  limitation  of  isotropic  potentials.  Allowing  the  potentials  to  have 
angular  dependence  leads  to  the  robust  formation  of  high-quality  lattices  from  initial  conditions 
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with  large  viiriations  in  density.  Moreover,  admitting  potentials  with  angular  dependence  allows  for 
the  construction  of  potentials  that  form  more  exotic  lattices,  such  as  the  kagome  lattice,  which  has 
not  yet  been  accomplished  with  purely  isotropic  potentials  [23],  Certainly,  the  use  of  anisotropic 
potentials  is  well- motivated  by  abundant  natural  examples  of  anisotropic  interaction  potentials  in 
Nature  -  the  water  molecule  serving  as  an  ubiquitous  prototype. 

In  order  to  introduce  potentials  with  angular  dependence,  we  must  first  extend  the  config¬ 
uration  space  of  the  particle  system  to  include  an  angular  coordinate.  We  no  longer  consider 
the  particles  as  point  particles,  but  rather  as  two-dimensional  sliding  disks,  each  with  radius  R 
and  uniform  mass  density.  The  configuration  manifold  of  each  particle  is  now  x  5^  The 
configuration  of  the  particle  is  described  by  the  coordinate  chart  0,)  as  indicated  in  Fig¬ 

ure  13.  and  we  write  the  vector  of  coordinates  describing  the  configuration  of  all  N  particles  as 
X  =  ,  (xa^, 2/Ar, ^iv)].  The  total  interaction  potential  over  all  particles  now  has  the 


FIG.  13:  To  implement  an  anisotropic  potential,  partieles  are  now  modeled  as  two-dimensional  disks  whose  eon- 
figuration  is  deseribed  by  a  location  for  the  center  of  the  disk,  (x,  y),  as  well  as  a  heading  angle  6.  The  interaction 
potential  between  tw^o  particles  is  a  function  of  the  distance  between  the  particles,  as  well  as  the  angle,  a,  that  lies 
between  the  heading  angle  and  the  bearing  toward  the  second  particle. 

form 

N 

F(x)  =  ^  ^  Fpajr(xi  “  Xj,i/i  —  yj^Oi^Oj)  (8) 

i<j 

where  Vpair  *.  R  x  R  x  5^  x  5^  — >  R  is  the  pairwise  interaction  potential  between  particles  that 
depends  not  only  on  the  relative  displacement  between  particles,  but  also  the  angular  displacement 
of  each  particle  relative  to  the  angular  bearing  of  the  other  particle.  By  construction,  we  make 
Vpair  symmetric  with  respect  to  particle  interchange.  Specifically, 

-  Xj,  j/,  -  yj,9„0j)  ■■=  ip(xi  -  Xj,yi  -  yj,0i)  +  V'{xj  -  xuyj  -  yi,0j) .  (9) 
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It  remains  to  choose  the  functional  dependence  of  to  reflect  the  desired  symmetry  in  the 

target  lattice.  Consider  the  interaction  potential 

4,{Ax,  Ay,  Ok)  ••=  2^  -  ^  [l  -  (^)]  (10) 

where 

r  :=  \/(Aj:)2  +  (Ay)2  (11) 

is*  the  radial  distance  between  the  particles,  and 

Oik  •=  ^k  ~  arctan(Ay,  Ax)  (12) 

is  the  angle  between  the  heading  of  particle  k  and  the  bearing  toward  the  second  particle  (see  Figure 
13.)  This  potential  is  simply  a  Lennard-Jones  potential  with  added  amplitude  modulation  in  the 
azimuthal  direction  on  the  attractive  term.  The  periodicity  of  the  trigonometric  functions  ensures 
that  the  potential  has  n-fold  radial  symmetry,  and  that  each  particle  has  preferred  directions  along 
which  it  feels  the  attractive  pull  of  neighboring  particles.  The  free  parameter  u  is  chosen  to  adjust 
the  shape  of  the  potential.  VV^hen  u  is  zero,  the  potential  collapses  to  the  isotropic  Lennard-Joncs 
potential.  For  values  of  between  zero  and  unity,  the  potential  has  n  potential  wells  symmetrically 
distributed  in  the  azimuthal  direction.  Taking  values  of  greater  than  unity  raises  the  repulsive 
regions  between  the  potential  w'ells.  Hence,  is  a  parameter  that  determines  how  strongly  the 
anisotropic  potential  prefers  the  binding  site  directions.  In  simulations,  we  have  observed  that 
taking  larger  values  of  i/  produces  lattices  with  less  defects  since  particles  that  do  not  align  with 
the  preferred  binding  directions  fall  into  these  more  repulsive  regions  betw'een  the  wells  and  create 
a  configuration  with  much  higher  energy.  These  defects  are  quickly  removed  by  vibrations  in  the 
lattice.  In  practice,  a  compromise  must  be  met  since  the  larger  regions  of  repulsion  created  by 
higher  values  of  u  increase  the  time  taken  for  self-assembly  to  occur  -  particles  only  bind  wdth 
one  another  if  they  approach  each  other  along  an  ever  more  narrower  binding  direction.  In  the 
simulatioTLs  that  follow,  we  have  used  a  value  of  r/  =  1 .5. 

By  changing  the  integer  value  of  n,  w'e  can  induce  the  formation  of  lattices  with  desired  n-fold 
symmetry.  Surface  plots  of  potentials  with  3-fold  and  4- fold  symmetry  (n=3  and  n=4)  as  a  function 
of  the  radial  coordinate  r  and  the  azimuthal  angle  a,  are  provided  in  Figures  14(a)  and  14(b).  A 
honeycomb  lattice  and  a  square  lattice  produced  with  these  potentials  using  particles  initialized  at 
low  density  are  shown  in  Figures  14(c)  and  14(d). 

For  the  creation  of  more  exotic  lattices,  we  alter  the  azimuthal  modulation  of  the  potential 
even  further.  Each  particle  in  the  kagome  lattice,  for  example,  has  binding  sites  at  angles  of  0"^, 
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(a) 


(b) 


♦♦♦♦♦♦♦♦♦♦ 

♦♦♦♦♦♦♦♦♦♦  ^ 
♦♦♦♦♦♦♦♦♦♦  ♦♦ 
♦♦♦♦♦♦♦♦♦♦♦♦♦ 
♦♦♦♦♦♦♦♦♦♦♦♦ 
♦♦♦♦♦♦♦♦♦♦♦♦ 


(c) 


(d) 


P  IG.  14:  The  potential  with  three-fold  symmetry  shown  in  (a)  yields  the  honeycomb  lattice  shown  in  (c).  Similarly, 
the  potential  with  four-fold  symmetry  shown  in  (b)  yields  the  square  lattice  shown  in  (d). 


60°,  180°,  and  240°  [49].  To  ensure  preferred  binding  along  these  directions,  we  must  modify  the 
azimuthal  dependence  of  the  Lennard-Jones  potential  accordingly.  Before  doing  so,  we  first  express 
the  potential  function  xp  of  line  (10)  in  the  following  equivalent  way: 

Ay,  Ok)  =  ^  ^S(Qk)]  (13) 

where 

5(o*)  :=sin*  .  (14) 

As  written  here,  xp  produces  a  potential  with  n-fold  symmetry.  In  order  to  produce  a  potential 
that  favors  the  kagome  lattice,  we  must  simply  introduce  a  new  definition  for  the  functional  form 
of5(.). 
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Let  B  :=  [6i,  •  •  •  , 6n]  denote  the  ordered  list  of  n  desired  binding  directions  measured  in  radians 
satisfying 


0  =  6i  <  •  •  •  <  6„  <  27r .  (15) 

Note  that  without  loss  of  generality,  we  may  prescribe  that  the  first  binding  direction  lies  along 
the  ray  corresponding  to  zero  radians.  For  the  honeycomb  lattice,  Bhc  :=  [O,  ^],  while  for  the 

kagome  lattice  we  have  Bkagome  :=  [0i  f  t]*  for  a  E  [0, 27r),  we  use  the  elements  in  the 

list  B  to  define  5(a)  piecewise  as  follows: 


f  sin^  ( \  tt) 

if  bi  <  a  <  6t+i  , 

for  i  =  1 ,  • 

■  •  ,  n  -  1 , 

(16) 

[sin' 

if  6n  <  a  <  27r . 

When  5(a)  defined  in  this  way  is  substituted  into  the  expression  for  the  anisotropic  potential 
function  ^  in  line  (13),  it  provides  the  necessary  azimuthal  modulation  of  the  Lennard- Jones 
potential  to  produce  potential  wells  along  the  binding  directions  specified  in  the  list  5.  A  plot  of 
5(a)  using  Bkagome  ii^  provided  in  Figure  15(a).  Notice  that  5(a)  has  local  minima  precisely  at  the 
binding  site  angles  of  0°,  60°,  180°,  and  240°  consistent  with  the  kagome  lattice  (recall  from  line 
(13)  that  minima  in  5(a)  lead  to  minima  in  the  interaction  potential).  Consequently,  the  angular 
dependence  in  the  resulting  interaction  potential  favors  the  bond  structure  peculiar  to  the  kagome 
lattice.  The  potential  and  a  lattice  resulting  from  this  potential  are  shown  in  Figures  15(b)  and 
15(c)  respectively. 

IX.  CONCLUSIONS  AND  FUTURE  WORK 

We  have  presented  and  compared  methods  for  generating  potentials  that  lead  to  the  self- 
assembly  of  specified  target  lattices.  In  particular,  we  have  addressed  the  problem  of  designing 
pairwise  interaction  potentials  that  induce  the  formation  of  a  honeycomb  lattice  when  a  planar 
system  of  particles  is  cooled. 

We  have  demonstrated  that  reasonably  high  quality  lattices  can  be  produced  using  a  heuristic 
computation-free  geometric  method.  The  geometric  method  provides  principles  that  utilize  purely 
geometric  information  to  design  potentials  that  by  construction  favor  and  stabilize  the  target 
lattice. 

A  trend  optimization  algorithm  has  been  introduced  that  quickly  and  robustly  finds  optimal 
shapes  of  the  interaction  potential  that  lead  to  the  self-assembly  of  lattices  of  high  quality.  The 
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(a)  (b) 


FIG.  15:  The  function  in  (a)  is  used  to  azimuthally  modulate  the  amplitude  of  a  Lennard- Jones  potential  to 
encourage  binding  directions  that  favor  the  kagomc  lattice.  The  resulting  potential  surface  is  shown  in  (b),  while  a 
kagome  lattice  formed  with  this  potential  is  shown  in  (c). 


success  of  the  trend  method  lies  in  its  ability  to  quickly  locate  minima  in  a  noisy  and  expensive 
objective  function.  Moreover,  the  potentials  discovered  using  the  trend  optimization  procedure 
robustly  form  high  quality  lattices  with  resp>ect  to  variations  in  the  initial  conditions  of  the  particles. 
\V^  have  seen  that  the  trend  optimization  method  has  discovered  a  family  of  potentials  characterized 
by  very  simple  exponential  decay  profiles  that  routinely  lead  to  the  formation  of  the  honeycomb 
lattice. 

Our  trend  optimization  algorithm  robustly  and  routinely  finds  optimal  values  of  the  objective, 
although  it  must  be  noted  that  as  currently  implemented,  the  algorithm  does  not  provide  rigorous 
guarantees  of  convergence.  Convergence  can  be  guaranteed,  however,  by  incorporating  a  polling 
step  as  required  in  the  Surrogate  Management  Framework.  It  would  be  interesting  to  investigate 
the  effects  of  polling  on  the  optimal  results  and  the  efficiency  of  the  method.  One  expects  that  the 
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addition  of  a  polling  step  to  guarantee  convergence  will  represent  only  a  marginal  increeu^  in  the 
overall  computational  cost  of  the  algorithm. 

The  geometric  method  ha5  also  been  extended  to  the  design  of  anisotropic  potentials.  Azimuthal 
dependence  of  the  interaction  potential  allows  for  the  formation  of  the  kagome  lattice  which  has  not 
previously  been  performed  using  isotropic  potentials.  Incorporating  anisotropy  into  the  potentials 
allows  for  the  formation  of  lattices  over  a  wide  range  of  particle  densities. 

An  auxiliary  contribution  of  this  paper  is  the  development  of  two  metrics  for  objective  analysis 
of  lattice  quality.  The  development  of  these  metrics  was  necessary  for  comparison  of  the  lattices 
produced  by  the  proposed  potentials. 

We  gmticipate  that  the  methods  presented  here  will  naturally  extend  to  three  dimensions  without 
impediment.  Rechtsman  et  al  have  recently  investigated  the  design  of  potentials  for  self-ai>sembly 
of  three-dimensional  lattice  structures  [51].  Most  notably,  they  have  demonstrated  the  formation  of 
the  diamond  and  wurtzite  lattices.  As  a  matter  of  course,  we  intend  to  apply  the  methods  presented 
here  to  the  design  of  potentials  in  the  three-dimensional  self-assembly  problem  and  expect  the  trend 
optimization  method  to  provide  a  significant  computational  savings  in  the  design  of  potentials.  Of 
particular  interest  is  to  determine  if  trend  optimization  recovers  the  three-dimensional  result  of  [51], 
or  if  a  new,  simpler,  and  perhaps  more  robust  family  of  solutions  is  discovered. 

A  natural  extension  of  the  methods  presented  is  to  investigate  the  use  of  multi-specie  and  multi¬ 
body  potentials  for  the  self-assembly  of  quasicrystals  and  Penrose  tilings.  Generating  lattices  with 
prescribed  geometry  and  structure  is  highly  motivated  by  the  desire  to  produce  photonic  crystals 
with  specified  optical  properties.  Also,  wc  intend  to  explore  the  use  of  the  trend  optimization 
method  for  designing  potentials  that  lead  to  the  the  formation  of  hexahedral  meshes  over  compli¬ 
cated  three-dimensional  volumes.  Producing  hexahedral  meshes  is  a  notoriously  difficult  problem 
that  is  currently  a  major  stumbling  block  for  continuum  mechanics  computations  that  use  a  finite 
element  method. 

More  broadly,  we  expect  to  use  trend  optimization  to  understand  more  deeply  the  fundamental 
limitations  and  extraordinary  possibilities  of  self-assembly.  It  will  be  interesting  to  investigate, 
for  example,  the  role  of  optimal  potential  design  in  natural  systems.  In  this  regard,  a  study  of 
self-ai>sembly  is  a  study  of  life  itself.  Are  there  mechanisms  at  the  level  of  local  interactions  that 
allow  for  the  differentiation  of  cells  that  self- assemble  to  form  bone,  for  instance,  from  those  that 
form  brain  tissue?  How  much  control  authority  over  the  superstructure  formed  via  self-assembly  is 
provided  by  control  over  the  local  interactions?  Are  there  natural  systems  in  which  a  small  amount 
of  flexibility  in  the  properties  of  the  local  interactions  allows  for  large  and  beneficial  changes  in 
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structure  at  the  macroscopic  level?  We  believe  that  insights  to  these  questions  may  be  afforded  by 
finding  optimal  potentials  through  the  method  of  trend  optimization. 
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Abstract 

Motivated  by  the  work  in  [1]  on  design  of  potentials  for  spontaneous 
self-assembly  of  particle  systems  into  2D  target  structures,  we  propose 
three  tools  to  apply  to  this  problem.  First,  we  introduce  a  new  pseudo¬ 
distance  between  planar  point  sets,  that  can  be  used  to  compare  particle 
configurations  with  the  target  structure.  In  conjunction  wdth  Molecular 
Dynamics  simulations,  it  provides  an  objective  function  to  be  graphed  or 
optimized  over  the  space  of  parameters  of  the  pjotential,  and  thus,  it  pro¬ 
vides  a  tool  that  quantifies  the  effectiveness  of  a  potential  to  assemble  a 
target  configuration.  The  second  tool,  is  the  use  of  infinitesimal  mechani¬ 
cal  stability  of  the  target  structure  with  a  given  inter-particle  potential  as 
a  necessary  condition  for  self-assembly.  The  third  one,  characterizes  the 
target  structure  (w'ith  given  inter-particle  potential  and  Rayleigh  dissipa¬ 
tion  forces)  as  an  asymptotic  fixed  point  of  the  system  dynamics  or  not, 
by  examining  the  behavior  of  initial  conditions  close  to  the  structure  in 
backward  time.  The  first  tool  provides  a  sufficient  condition  to  identify 
assembling  potentials,  but  it  can  be  computationally  expensive,  while  the 
other  tools  are  computationally  fast  and  provide  necessary  conditions  that 
can  be  used  to  exclude  a  priori  “bad”  regions  of  the  parameter  space. 


1  Introduction 

In  a  recent  paper  [1],  Rochtsman  et  al  discuss  “inverse  methods”  for  design  of 
potentials  for  self-assembly.  The  implied  “direct”  problem  is  the  determination 
of  the  minimum  energy  configuration  of  a  system  of  identical,  classical  interact¬ 
ing  particles  (at  a  fixed  density),  given  their  interaction.  The  “inverse”  problem 
is  to  design  the  interactions  between  the  particles,  so  that  their  minimum  en¬ 
ergy  configuration  (again,  at  fixed  density)  is  a  desired  one.  In  particular,  these 
authors  consider  isotropic  two-body  potentials  with  fixed  functional  form,  but 
tunable  parameters,  and  seek  to  determine  optimal  parameter  sets  so  that  a 
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particle  system  with  these  interactions  (in  2  dimensions)  would  “self-assemble’' , 
as  closely  as  possible,  into  a  (piece  of  a)  square  lattice  or  a  honeycomb  structure 
of  given  density,  as  the  temperature  decreases  to  T  =  0.  The  inspiration  for 
their  work  comes  from  a  host  of  examples  in  Biology,  Chemistry  and  Materi¬ 
als  Science  where  smaller  “units”  arrange  themselves  to  form  larger  structures 
through  their  interactions.  If  one  can  manipulate  these  interactions  to  some  ex¬ 
tent,  one  may  be  able  to  form  structures  with  desired  properties  and  functions, 
see  [1]  and  references  therein. 

The  general  methodology  of  [1]  is  as  follows:  Starting  with  a  two- body 
isotropic  potential  of  given  functional  form,  but  with  tunable  parameters  Ci, 
K(r,  {oj})  (r  is  the  inter- particle  distance),  they  propose  objective  functions  of 
the  Oi  whose  minimization  implies  that  the  particle  system  has  the  desired  struc¬ 
ture  (square  lattice  or  honeycomb  of  a  given  density)  as  its  ground  state,  and 
thus,  it  will  equilibrate  to  it  as  its  temperature  is  brought  down  to  0.  They  pro¬ 
pose  two  such  functions,  one  based  on  energy  considerations  and  one  based  on 
dynamical  ones  (“zero- temperature”  and  “near-melting”  schemes).  After  each 
of  the  optimization  schemes  is  terminated,  the  obtained  potential  is  tested  for 
effectiveness:  An  annealed  Monte  Carlo  (MC)  simulation  is  performed  in  an 
NVT  ensemble  starting  from  a  random  configuration,  and  the  resulting  mini¬ 
mum  energy  configuration  is  compared  to  the  desired  structure. 

In  this  work,  we  propose  some  tools  to  attack  the  problem  of  self-assembly. 
Before  we  introduce  these  tools,  we  want  to  comment  on  the  approach  of  [1]  and 
justify  the  need  for  our  methods.  As  pointed  out  by  the  authors,  the  energy- 
based  zero-temperature  scheme  ignores  all  other  competing  structures  other 
than  the  ones  it  explicitly  considers.  Though  physically  intuitive,  it  does  not 
provide  a  sufficient  or  necessary  condition  for  self-assembly.  The  near-melting 
scheme  offers  only  a  necessary  condition:  To  quote,  there  is  an  ” inherent  bias 
in  this  scheme  towards  the  target  lattice”,  meaning  that  the  scheme  guarantees 
non-local  stability  of  the  structure  but  not  self-assembly  from  any  (or  most) 
initial  conditions.  Also,  the  metric  used  by  the  authors  (and  referred  to  as 
the  ”Lindemann  parameter”)  to  compare  the  particle  system  configuration  near 
its  melting  point  with  the  target  structure,  is  not  entirely  satisfactory.  If  the 
particle  system  has  rotated  as  a  whole,  or  two  particles  have  exchanged  positions, 
the  distance  of  the  particle  configuration  from  the  target  will  be  increased. 
Similarly,  the  structure  assembled  by  the  final  MC  simulation  (to  check  the 
outcome  of  the  optimization)  is  qualitatively  compared  to  the  target,  either 
directly,  or  by  the  plot  of  the  structure  function  5(k),  see  section  2  and  [1]. 

In  our  approach,  we  seek  to  provide  clear-cut  necessary  and  sufficient  con¬ 
ditions  on  whether  a  set  of  parameters  renders  a  potential  capable  to  assemble 
a  structure.  The  first  tool  we  introduce  is  a  Fourier  space- based  distance  for 
point  sets  in  R^.  This  new  distance  function  allows  us  to  compare  two  planar 
point  sets  modulo  translations  and  rotations,  i.e.  a  point  set  and  a  translated 
and  rotated  copy  of  itself  are  considered  the  same.  It  allows  us  to  quantitatively 
compare  a  particle  configuration  with  a  target  structure,  rather  than  relying  on 
qualitative  estimations.  After  running  an  MD  simulation  of  the  particle  sys¬ 
tem  with  a  given  potential,  while  slowly  lowering  the  temperature  to  T  =  0  (the 
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freezing  ha«  to  be  slow  enough,  so  that  the  system  is  not  trapped  in  a  metastable 
state,  i.e.  a  local  minimum  of  the  interaction  energy),  we  calculate  the  distance 
of  the  equilibrium  configuration  from  the  target  structure.  Averaged  over  all  ini¬ 
tial  conditions,  this  provides  an  objective  function,  whose  minimum  determines 
the  best  set  of  parameters  (of  the  potential)  for  self-assembly  from  most  initial 
conditions.  We  will  refer  to  this  function  as  the  Defect  Metric.  In  practice,  see 
section  2,  it  suffices  to  consider  just  a  few  initial  conditions  per  function  eval¬ 
uation  (the  most  we  used  was  5).  Note  that  the  Defect  Metric  being  equal  to 
zero  (or,  at  least,  very  small)  for  a  set  of  parameter  values  provides  a  sufficient 
condition  for  self-assembly  of  the  target  structure.  We  consider  this  a  major 
advantage  of  this  method.  Also,  the  method  is  not  in  any  way  biased  towards 
the  target  structure.  We  will  not  attempt  any  optimization  in  the  examples 
considered  here,  but  rather  plot  the  Defect  Metric  in  parameter  neighborhoods 
around  the  solutions  of  [l]  in  order  to  illustrate  the  effectiveness  of  our  approach. 

The  second  tool  we  introduce,  is  the  use  of  the  (infinitesimal)  mechanical 
stability  of  a  structure  as  a  necessary  condition  for  its  self-assembly  from  a  given 
potential.  In  [1],  the  authors  emphasize  the  necessity  for  mechanical  stability 
of  a  target  structure  if  it  is  to  be  considered  as  a  ground  state  for  the  particle 
system  with  a  given  potential  interaction.  They  also  check  that  the  potentials 
furnished  by  their  optimization  schemes  guarantee  mechanical  stability  in  each 
case.  How'ever,  this  idea  can  be  exploited  further  to  delineate  portions  of  the 
parameter  space  where  the  resulting  potential  does  not  guarantee  mechanical 
stability  of  the  target  structure.  The  relevant  calculations  are  very  fast  and 
require  no  optimization.  This  provides  a  simple,  fast  test  that  can  drastically 
reduce  the  parameter  space  one  has  to  search  over  for  a  solution.  Note  that  a 
potential  that  guarantees  mechanical  stability  of  the  target  structure  does  not 
necessarily  assemble  it,  hence  this  condition  is  only  necessary. 

The  third  tool,  is  the  use  of  backward  (in  time)  integration  to  identify  an 
asymptotic  stable  fixed  point  of  a  dynamical  system.  It  is  based  on  the  property 
of  an  asymptotic  stable  fixed  point  that,  trajectories  slow  down  more  and  more 
the  closer  they  get  to  it.  Contrast  this  with  the  behavior  of  trajectories  around 
a  regular  point  of  phase  space,  where  they  don’t,  at  least  in  some  directions. 
The  idea  is  to  consider  initial  conditions  around  a  presumed  fixed  point  and 
study  their  behavior  in  reverse  time.  If  the  presumed  fixed  point  is  indeed  one, 
the  initial  conditions  will  move  away  from  it  slowly.  If  it  isn’t,  at  least  some 
of  them  will  move  away  from  it  fast  enough.  The  way  we  implement  this  idea 
in  the  self-assembly  problem,  is  the  following:  For  a  given  target  structure,  we 
create  an  ensemble  of  particle  configurations  uniformly  distributed  according 
to  their  distance  from  it  (with  the  Defect  Metric  taking  values  between  0  and 
a  small  upper  bound).  We  propagate  this  ensemble  backwards  in  time  for  a 
short  time  and  calculate  the  average  of  the  Defect  Metric.  A  large  value  of 
this  average  Defect  Metric  means  that  the  target  structure  is  not  an  asymptotic 
stable  fixed  point  for  the  potential.  As  with  mechanical  stability,  this  test 
provides  a  necessary  condition  only,  however,  it  is  fast  and  simple  to  implement, 
and  requires  no  optimization.  Again,  the  goal  is  to  exclude  portions  of  the 
parameter  space  and  thus  speed  up  the  search  for  good  parameter  sets. 
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We  will  use  three  examples  to  demonstrate  our  methods:  Self-ai>sembly  of 
a  square  structure  using  tw'o  candidate  potentials  with  parameters,  Kqui 

1  2 

Vaq»i{r;ao,aua2)  =  ^  ^  +  oq  exp[-ai  (r  -  Oq)^],  (1) 

Kiqu2(»';  01,02)  =  ^  +  tanh(oir-02)  -1,  (2) 

and  self-assembly  of  a  honeycomb  structure  using  Vhon^ 

Vhon(r;  00,01,02,03)  =  ^  ^  +  01  exp[-027']  -  A  exp[-40(r  -  03)^].  (3) 

Kjqui  and  Vhon  are  taken  from  [1],  while  Vsqu2  is  a  generalization  of  a  potential 

found  in  [2].  These  three  potentials  are  plotted  in  figure  1  for  parameter  values 
that  make  them  assembling.  The  physical  motivation  for  these  choices  of  func¬ 
tional  forms  is  the  following:  Ksqui  and  Vsqu2  energetically  favor  neighbors  at 
distances  1  and  y/2  (first  and  second  neighbors  in  the  case  of  a  square  lattice  of 
lattice  constant  1)  while  they  strongly  disfavor  neighbors  at  distance  (second 
neighbors  in  the  case  of  a  triangular  lattice  of  constant  1,  the  triangular  lattice 
being  the  main  competitor  of  any  2D  structure  auisembled  by  an  isotropic  poten¬ 
tial).  Vhon  energetically  favors  neighbors  around  r  =  \/3.  It  disfavors  neighbors 
around  r  =  1,  but  makes  them  locally  stable  (see  small  dip  in  Vhon  close  to 
r  =  1;  the  actual  lattice  constant  of  the  honeycomb  structure  for  this  example 
is  1.0565).  At  the  right  density,  this  competition  between  energy  and  mechani¬ 
cal  stability  favors  the  honeycomb  structure  which  has  only  3  first  neighbors  at 
distance  1.0565  versus  the  triangular  lattice  that  has  6  first  neighbors  at  this 
distance  (they  both  have  6  second  neighbors  at  distance  y/3  x  1,0565). 

The  organization  of  the  rest  of  the  paper  is  as  follows:  In  section  2,  we 
introduce  our  Fourier  space- based  distance  of  point  sets  in  and  use  it  to 
compute  the  Defect  Metric  as  a  function  of  the  potential  parameters  in  each 
example.  In  sections  3  and  4,  we  present  our  mechanical  stability  and  backward 
integration  calculations,  respectively.  Section  5  concludes.  Some  properties  of 
Fourier  space-based  distance  functions  of  point  sets  in  R^  are  discussed  in  an 
Appendix. 

2  Defect  Metric 

In  this  section  we  introduce  a  new  metric  that  compares  point  configurations 
(sets)  in  R^  modulo  translations  and  rotations.  It  allows  us  to  compare  the 
‘^shapes’'  of  two  point  sets  as  structures,  without  the  need  to  use  an  a  priori 
correspondence  of  points  of  the  two  sets.  The  use  of  this  metric,  combined  with 
Molecular  Dynamics  simulations  will  provide  a  sufficient  test  for  the  assembling 
capability  of  potentials.  Note  that  the  constructions  in  this  section  (as  well  as 
the  derivations  in  the  Appendix)  generalize  immediately  to  R"'. 
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Figure  1:  Potentials  14qui,  V^qu2  and  Vhon  for  parameter  values  that  make  them 

aSvSembling. 


Consider  a  configuration  of  points  {rj},  i  =  in  E?.  Assigning  a 

mass  of  1  to  each  point,  we  may  define  the  (singular)  mass  distribution  of  this 
point  set  to  be 

N 

Pi^)  =  r<). 

1  =  1 

The  Fourier  transform  of  p  is  given  by 

N 

c(k)  =  keK^  (4) 

t  =  l 

The  c(k)’s  were  introduced  in  [3]  under  the  name  collective  coordinates,  with  the 
purpose  of  capturing  collective  aspects  of  the  dynamical  behavior  of  a  system  of 
identical  particles  .  For  some  of  their  uses,  see  [4,  5,  6]  and  references  therein. 
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The  quantity  |c(k)p  is  the  Fourier  transform  of  the  pair  correlation  function  of 
the  mass  distribution  of  this  point  set,  i.e. 


I 


,ik-r 


|c(k)|2  =  ^<5(r-(r,-r,-)). 


Notice  that  it  is  invariant  to  translations  of  the  point  set,  Cj  ->  r,-  +  c.  Since 
the  pair  correlation  function  of  a  finite  point  set  (i.e.  the  knowledge  of  the  set 
of  relative  positions  {r^  —  r^})  determines  the  point  set  modulo  translations, 
so  does  the  structure  function  5(k)  =  \c(k)\^/N.  Note  that  this  fact  does 
not  resolve  the  Crystallographic  Phai>e  Problem  in  general,  i.e.  a  distribution 
cannot  be  reproduced  by  knowledge  of  the  moduli  of  its  Fourier  coefficients  only, 
but  for  the  restricted  class  of  distributions  we  are  considering,  namely  finite 
superpositions  of  delta  measures,  knowledge  of  |c(k)p  is  enough  to  determine 
the  point  set  modulo  translations. 

Given  two  point  sets,  {r-^^}  and  i  =  1, . . . ,  N,  we  define  a  distance 

d{{r^^^},  between  them  (modulo  translations)  by 

d{{r<')},  {r(2)})  =  1 1  ^u.(k)  ||c(i)(k)|2  -  |c(2)(k)p| ,  (5) 


where  w  is  a  positive,  continuous  and  integrable  function  that  weighs  the  rela¬ 
tive  importance  of  Fourier  modes.  In  the  Appendix,  it  is  shown  that  d  is  well 
defined,  and  satisfies  the  properties  of  a  distance  function.  Different  choices 
of  w,  emphasize  different  spatial  characteristics  of  the  point  set.  Consider,  for 
example,  the  point  sets  a  and  b  of  figure  2:  a  is  a  piece  of  square  lattice  of 
constant  1  made  up  of  10  x  10  sites  and  b  is  made  by  shifting  the  left  half  of  a 
(10  X  5  sites)  by  1.  Structures  a  and  b  appear  much  the  same  in  their  bulk  (i.e. 
away  from  their  boundaries),  but  globally  the  sets  are  distinct.  We  can  capture 
the  similarities  and  the  differences  of  these  two  sets  at  different  length  scales  by 
using  different  weight  functions.  For  example, 


wi{k)  = 


1,  kx  S  (— IOtt,  IOtt],  ky  G  [— IOtt,  IOtt], 
0,  otherwise. 


penalizes  differences  in  the  structures  in  length  scales  from  .2  and  up,  and  gives 
a  value  of  d  =  17  for  the  distance  of  a  and  b. 


tt^2(k)  = 


1,  Ati  6  [— IOtt,  — tt]  U  (tt,  IOtt],  /Cy  G  (—IOtt, —tt]  U  [tt,  IOtt], 
0,  otherwise. 


penalizes  differences  in  length  scales  from  .2  up  to  2  only  (i.e.  inside  a  5  x  5  box 
centered  at  every  lattice  site)  and  thus  gives  a  much  smaller  value  of  d  =  1.8 
(the  calculations  are  done  using  a  discrete  version  of  (5)  with  a  and  b  inside  a 
20  X  20  box).  So,  w  provides  a  flexibility  in  the  comparison  of  characteristics  of 
two  point  sets  at  different  scales. 
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a 


b 


Figure  2:  Left:  A  piece  of  square  lattice.  Right:  The  left  half  of  a  is  shifted  up 
by  one  lattice  constant. 


To  compare  the  “shapes”  of  two  point  sets,  one  needs  to  mod  out  not  only 
translations,  but  also  rotations.  To  this  end,  we  define  the  quantity 


(6) 


The  crucial  step  here  is  the  angular  k-integration  that  eliminates  all  information 
in  the  structure  function  except  for  the  pairwise  distances  in  the  point  set.  In 
the  Appendix,  it  is  shown  that 


where  Vij  =  ||ri  —  rj||,  and  hence  I(s)  only  retains  information  about  the  pair¬ 
wise  distances  of  points.  It  is  shown  in  [7,  Theorem  2.6]  that  the  distribution 
of  pairwise  distances  of  points  in  a  point  configuration  uniquely  determines  it 
modulo  translations  and  rotations,  except  for  a  set  of  configurations  of  measure 
zero  in  the  space  of  all  possible  point  configurations. 

We  define  now  the  following  pseudo-distance  function  between  two  two  point 


sets,  {r|*^}  and  i  =  1, . . . , 


w  must  be  a  positive,  continuous  and  integrable  function  in  [0,  oo).  In  the 
Appendix,  we  show  that  u  is  well-defined,  and  it  is  a  pseudo-distance.  As  in 
(5),  w  weighs  various  length  scales  differently.  Consider,  for  example,  the  point 
sets  a  and  c  in  figure  3  (c  is  made  from  a  by  slightly  shifting  and  rotating  its 
upper  right  quarter).  For  u?i(s)  =  1,  s  6  [0,  IOtt],  their  distance  is  u  =  0.0038. 
Using 


7 


281 


C.2.  TOOLS  FOR  DESIGN  OF  POTENTIALS  FOR  PARTICLE  SELF-ASSEMBLY 


reduces  the  distance  to  u  =  0.0011.  As  expected,  when  the  large  scale  features 
are  ignored,  the  difference  between  a  and  c  is  much  less  pronounced. 


a 


c 


Figure  3:  Left:  A  piece  of  square  lattice.  Right:  The  upper  right  quarter  of  a 
is  slightly  shifted  and  rotated. 


In  each  of  the  three  examples  mentioned  in  the  introduction  (two  candi¬ 
date  potentials  for  the  square  lattice  and  one  for  the  honeycomb  structure),  we 
performed  MD  simulations  for  particles  in  a  box  with  periodic  boundary  condi¬ 
tions,  with  density  fixed  to  the  appropriate  value  for  each  target  structure.  The 
simulations  were  performed  using  the  MD  softw’are  package  LAMM  PS  (Large- 
scale  Atomic/Molecular  Massively  Parallel  Simulator)  [8].  The  initial  positions 
of  the  particles  formed  a  square  lattice  and  the  initial  velocities  were  randomly 
generated.  The  system  temperature  was  slowly  decreased  close  to  T  =  0  using  a 
Nose-Hoover  thermostat  [9,  10,  11].  The  Defect  Metric  (distance  u  of  the  target 
structure  from  the  final  configuration  of  the  MD)  was  computed  for  a  grid  of 
parameters  in  each  case,  using  a  small  number  of  runs  per  evaluation  (average 
over  runs).  The  weight  function 


w(s) 


0,  s  <  irfly 
1,  s  >  tt//. 


(/  being  the  lattice  constant  in  each  case)  was  used  in  its  computation,  so  that 
large-scale  differences  between  the  achieved  structures  and  the  target  were  not 
penalized.  The  integral  (6)  was  replaced  with  the  sum  over  Fourier  modes 
appropriate  to  the  container  box  of  each  simulation. 

The  results  of  the  simulations  are  plotted  in  figure  4.  The  top  plot  rep¬ 
resents  the  Defect  Metric  as  a  function  of  parameters  ai  and  a2  of  V^qui, 
with  ao  set  equal  to  0.828.  It  is  immediately  seen  that  the  parameter  val¬ 
ues  (ai,a2)  =  (26.5,1.79)  of  (1)  for  Kqui  fall  exactly  in  the  parameter  region 
where  the  Defect  Metric  is  close  to  zero.  The  middle  plot  contains  the  Defect 
Metric  for  K,qu2-  As  a  side  remark,  compare  the  two  wedge-like  domains  of  the 
parameter  space  where  the  Defect  Metric  is  (approximately)  zero  for  V4qu2  with 
the  corresponding  narrow  strip  of  parameter  values  for  V^qui-  It  is  obvious  that 
Ksqu2  is  more  robust  to  changes  in  its  parameters  than  Kjqui-  Finally,  for  VhonT 
the  parameters  (ao,a2)  =  (5.89,2.49)  of  [1]  fall  into  a  shallow  and  narrow  min¬ 
imum  of  the  Defect  Metric,  see  bottom  plot.  As  stated  in  the  Introduction,  we 
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did  not  attempt  any  optimization  here,  but  rather,  we  wanted  to  demonstrate 
the  effectiveness  of  our  approach.  If  desired,  optimization  over  the  parameters 
Ui  can  be  done  using  simulated  annealing  as  in  [1],  or  other  methods,  such  as 
trend  optimization  [12,  13]  that  are  very  effective  when  one  is  optimizing  over 
functions  that  are  expensive  to  evaluate  and  noisy  (as  is  the  case  here,  because 
the  objective  function  is  evaluated  using  a  few  MD  runs  with  different  initial 
conditions).  We  plot  the  same  results  in  the  top  plots  of  figures  7-9,  along  with 
the  results  of  sections  3  and  4  for  easy  comparison. 


Oo 


Figure  4:  Top:  Defect  Metric  as  a  function  of  parameters  Ui  and  02  of  V^qui, 
with  oo  =  0.828.  Middle:  Defect  Metric  as  a  function  of  parameters  ai  anda2 
of  I4qu2-  Bottom:  Defect  Metric  as  a  function  of  parameters  ao  and  02  of  Vhonj 
with  Oi  =  17.9  and  03  =  1.823.  The  stars  correspond  to  the  solutions  of  [1] 


3  Mechanical  stability 

Infinitesimal  mechanical  stability  of  an  equilibrium  configuration  of  a  particle 
system  is  a  test  that  the  said  configuration  is  a  (local,  in  general)  minimum  of 
the  potential  interaction  energy.  Hence,  it  is  a  natural  condition  to  check  in  the 
search  for  the  global  energy  minimum.  Any  potential  (and  hence,  set  of  param¬ 
eters)  that  renders  the  desired  structure  infinitesimally  mechanically  unstable, 
cannot  have  that  structure  as  a  ground  state.  This  provides  a  simple  and  fast 
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test  that  excludes  portions  of  the  parameter  space  before  any  search  or  opti¬ 
mization.  The  quantity  to  be  calculated,  is  the  Hessian  of  the  potential  energy 
of  the  system,  evaluated  at  the  equilibrium  configuration.  Semi-definiteness  of 
the  Hessian  is  equivalent  to  mechanical  stability.  For  infinite  periodic  config¬ 
urations  of  identical  particles,  the  eigenvalues  of  the  Hessian  are  the  squared 
frequencies  of  the  phonon  excitations  (modes)  of  the  structure. 

We  plot  the  minimum  squared  frequency  of  the  phonon  modes  of  the 
target  structure,  as  a  function  of  the  parameters  in  the  potential  in  each  case, 
in  figure  5.  When  the  minimum  is  negative,  the  potential  renders  the  target 
structure  infinitesimally  mechanically  unstable.  For  the  case  of  the  square  lattice 
with  Fsqui  and  I4qu2j  the  top  and  middle  plots  of  figure  5  delineate  the  “a  priori 
good’'  portions  of  the  parameter  space  with  with  very  good  accuracy.  In  the  case 
of  the  honeycomb  structure  with  Vhon>  the  bottom  plot  of  figure  5  shows  that 
about  30%  of  the  parameter  space  depicted  can  be  excluded  beforehand.  The 
same  results  are  plotted  in  the  middle  color  plots  of  figures  7-9  for  comparison 
with  the  results  of  sections  2  and  4. 


(k) 

Figure  5:  Top:  Minimum  squared  phonon  frequency  in  a  square  lattice  of 
constant  1  as  a  function  of  parameters  ai  and  Uq  of  \4qui,  with  ao  =  0.828. 
Middle:  Minimum  squared  phonon  frequency  in  a  square  lattice  of  constant 
1  as  a  function  of  parameters  aj  and  a2  of  Vsqu2-  Bottom:  Minimum  squared 
phonon  frequency  in  a  honeycomb  structure  of  constant  1.0565  as  a  function 
of  parameters  ao  and  a2  of  V^onj  with  ai  =  17.9  and  aa  =  1.823.  The  stars 
correspond  to  the  solutions  of  [1] 
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4  Backward  integration 

Consider  a  dynamical  system  with  a  globally  attractive  fixed  point  and  a  small 
neighborhood  around  that  point.  If  we  reverse  the  arrow  of  time,  this  small 
neighborhood  will  expand,  and  in  the  limit  t  — >  — oo,  it  will  encompass  the 
whole  space.  This  is  an  equivalent  way  to  state  that  the  fixed  point  is  globally 
attractive.  Notice,  that  this  does  not  happen  for  regular  points  in  the  phase 
space:  A  neighborhood  of  a  regular  point  will  be  mapped  only  to  a  subset  of 
the  phase  space  as  <  — >  — oc.  This  idea  could  be  used,  in  principle,  in  the 
self-assembly  problem  in  the  following  way:  If  a  target  structure  is  to  be  the 
ground  state  of  a  particle  system  for  a  given  potential,  then  a  a  neighborhood 
of  initial  conditions  (particle  positions/velocities)  around  the  target  structure 
at  equilibrium  would  expand  to  the  full  phase  space  of  the  particle  system  in 
the  limit  t  — >  — oo.  However,  for  an  N  particle  system  in  2D,  this  would  require 
propagating  a  4 TV- dimensional  probability  distribution  function,  a  daunting  task 
even  for  small  N.  Instead,  we  use  a  different  manifestation  of  the  asymptotic 
stability  of  a  fixed  point  in  backward  evolution.  Consider  an  asymptotic  stable 
fixed  point  (not  necessarily  globally  attractive).  Initial  conditions  in  a  small 
neighborhood  of  it,  will  move  away  from  it  when  we  reverse  time  evolution,  but 
very  slowly.  This  is  equivalent  to  the  asymptotic  approach  of  trajectories  to  the 
fixed  point  in  forward  evolution.  This  “slowing  down”  of  trajectories,  however, 
does  not  happen  for  regular  points  in  phase  space,  no  matter  how  close  we  get 
to  that  point.  Hence,  in  reverse  time,  initial  conditions  around  a  regular  point, 
tend  to  move  away  from  it  faster  (at  least  some  of  them).  We  use  this  fact 
to  construct  a  simple  test  for  detecting  non- assembling  potentials:  For  a  given 
target  structure,  we  create  an  ensemble  of  50  particle  configurations  uniformly 
distributed  according  to  their  distance  from  it.  We  propagate  this  ensemble 
backwards  in  time  for  a  short  time  and  calculate  the  average  of  the  Defect 
Metric.  Large  magnitudes  of  this  average,  signify  non-assembling  potentials.  As 
with  mechanical  stability,  this  test  provides  a  necessary  condition  only,  however, 
it  is  fast  and  simple  to  implement,  and  requires  no  optimization.  The  goal,  again, 
is  to  exclude  portions  of  the  parameter  space  and  thus  speed  up  the  search  for 
good  parameter  sets. 

Figure  6  contains  the  plots  of  the  average  Defect  Metric  for  the  particle 
ensembles  that  were  integrated  backwards  for  a  short  time,  for  each  example, 
as  functions  of  the  potential  parameters.  The  same  results  are  plotted  in  the 
bottom  color  plots  of  figures  7-9  for  comparison  with  the  results  of  sections  2 
and  3.  Notice  the  very  good  agreement  with  the  mechanical  stability  plots. 

5  Conclusion 

Motivated  by  the  recent  work  [1],  we  presented  three  new  tools  for  optimizing 
potentials  for  self-assembly.  The  first  tool,  is  a  new  (pseudo)  metric  for  compar¬ 
ison  of  particle  configurations.  In  conjunction  with  MD  simulations,  it  provides 
an  automated  sufficient  criterion  for  detection  of  good  parameter  values  of  the 
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Figure  6:  Top:  Average  Defect  Metric  ay  a  function  of  parameters  ai  and  02 
of  Fgqui,  with  tto  =  0.828.  ^Middle:  Average  Defect  Metric  as  a  function  of 
parameters  a\  and  02  of  V4qu2*  Bottom:  Average  Defect  Metric  as  a  function 
of  parameters  ao  and  02  of  Vhon»  with  ai  =  17.9  and  03  =  1.823.  The  stars 
correspond  to  the  solutions  of  [1] 


potentials.  Since  it  is  based  on  Fourier  space  considerations,  it  is  ideal  for  target 
structures  with  simple  spectra,  like  crystals  and  quasi-crystals  (see  [14]  for  a  re¬ 
cent  work  on  assembly  of  crystals  and  quasi-crystals  with  a  simple  double-well 
potential).  The  other  two  tools,  infinitesimal  mechanical  stability  and  backward 
integration,  provide  simple  (no  optimization  required)  and  fast  (no  MD’s  or  very 
short  run  MD’s  needed)  criteria  that  can  a  priori  exclude  “bad”  portions  of  the 
parameter  space  to  be  searched.  This  makes  them  invaluable  when  the  potential 
depends  on  3  or  more  parameters. 
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Appendix 

In  this  appendix,  we  present  proofs  that  the  functions  d  and  u  defined,  respec¬ 
tively,  in  (5)  and  (7),  are  metrics  (actually,  u  is  a  pseudometric).  From  the 
definition  (4),  it  is  immediately  seen  that  |c(k)|  <  N.  We  have,  for  d: 

1.  is  well-defined: 

d({r(»},{r(^)})  =  iy^u.(k)||c(»(k)|^-|c(^)(k)|^| 

r  d^Xc 

2.  d  is  symmetric  in  its  arguments  (obvious) 

3.  d  satisfies  the  triangle  inequality: 

d({r(')},{r(3)}) 

=  ^/  ^tx>(k)||c<‘)(k)|^-|c<^)(k)|^| 

=  rf({r(')},  {r'^)})  +  d({r'^^},  {r'^*}). 

4.  If  d({r^'^},  {r^^^})  =  0,  the  two  point  sets  are  identical  up  to  a  global 
translation. 


d({r<’)},{rW})  =  0 
=>  |c'’>(k)|2-|c'^)(k)|2  =  0,  VkeR^ 

=>  /  -  k'")(k)|2)  =  0 

N  N 

i^j  m^n 

which  means  that  the  relative  positions  in  the  first  point  set  are  the  same 
as  in  the  second  point  set,  including  multiplicities.  This  implies  the  state¬ 
ment,  i.e.  the  second  point  set  is  just  a  translation  of  the  first. 

Now,  for  u: 
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1.  {r^^^})  is  well-defined: 

Prom  the  definition  of  /(s),  we  have  that  I{s)  <  N.  Then, 


2. 

3. 


rOO 

u({r(')},{r(^)})  =  /  w{s)\h{s)-  l2{s)\ds 

Jo 

roo 

<  /  w{s){Ii{s)  +  l2{s))ds 

Jo 


<  2N 


rOO 

I  w{s)ds 
Jo 


<  00. 


u  is  symmetric  in  its  arguments  (obvious) 
u  satisfies  the  triangle  inequality: 

u({r<')},{r(3)}) 

rOO 

=  /  w{s)\Ii{s)  -  h{s)\ds 

Jo 

rOO 

<  /  W{s){\h{s)-l2{s)\  +  \l2{s)-h{s)\)ds 

Jo 


4.  If  =  0,  the  pairwise  distances  in  the  first  point  set  are 

the  same  as  in  the  second  point  set,  including  multiplicities.  This  implies 
[7]  that  the  point  configurations  are  identical  modulo  translations  and 
rotations,  except  for  a  set  of  configurations  of  measure  zero  in  the  space 
of  all  possible  point  configurations.  Hence  u  is  only  a  pseudometric. 

First,  we  compute  I{s)  analytically.  We  have, 


l|k||5 

1  r  kdk  1 


Ntts^  , 

Jo  27r 

1 

r  kdk 

Ntts^  . 

Jo  27r 

[Note  : 

n-rj 

1 

r  kdk 

Ntts^  , 

Jo  27r 

1 

Y'  1 

2Nn'^s 

1 

sr  1 

r  kdk  1  ,,  .  ,.,,2 

/  — — —  /  a0|c(/i:cos0x  +  fcsin  0y)| 

Jo  27r  27r  Vq 


p-ikrij  cos{4>-<i>ij) 


J\  {rjjs) 


2Nn'^s'^  ^  Tj, 


2A^7r2  ^  niS 
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Then, 


«({r(')},{rW})  =  0 
/l(s)  =  /2(s)  Vs  e  [0,  oo) 

^  e  [O.oo) 


1  rOo 

1  yoo 

H  157  / 

n#m  ‘^0 


n^m  ^nm 


which  implies  that  the  pairwise  distances  in  the  first  point  set  are  the  same 
as  in  the  second  point  set,  including  multiplicities. 
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Correct  and  fast  computation  of 
phase  diagrams  in  the  presence  of 
uncertainty 

D.l  Constructing  the  phase  diagram  for  krypton  on  graphite  by 
detecting  pattern  boundaries 
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1  Introduction 

The  phase  diagram  for  krypton  atoms  on  a  graphite  substrate  can  be  constructed 
by  running  molecular  dynamics  simulations  at  a  variety  of  temperature  and  density 
conditions,  and  observing  some  measure  of  the  order  of  the  system.  Rather  than 
using  such  an  order  parameter  for  the  system  as  a  whole,  the  approach  taken  here 
is  to  focus  on  local  properties.  In  particular,  the  density  of  the  different  phases  that 
are  in  co-existence  in  our  system. 

Using  snapshots  from  molecular  dynamics  simulations,  the  fluid  and  solid  regions 
are  identified  based  on  the  local  density  of  particles.  The  overall  density  in  these 
regions  is  then  calculated  and  this  yields  points  on  the  phase  co-existence  curve 
at  the  temperature  of  the  snapshot.  The  phase  diagram  can  be  constructed  by 
analysing  snapshots  from  different  final  temperatures  in  this  way. 

In  order  to  calculate  the  extent  of  the  fluid  region,  the  boundary  between  the 
fluid  and  solid  phases  needs  to  be  accurately  identified.  This  is  achieved  by  looking 
not  only  at  the  local  density  of  particles  but  also  at  the  local  density  around  nearest 
neighbour  particles. 

The  motivation  for  this  approach  is  Gclb  and  Muller’s  temperature  quench 
method  [1]  described  in  the  following  section. 

The  results  of  the  pattern  boundary  detection  method  will  be  compared  with  the 
phase  diagram  for  krypton  on  graphite  that  was  found  by  Vladimir  Fonoberov  using 
molecular  dynamics  simulations.  They  will  also  be  compared  against  experimental 
results  due  to  Butler  [Butler]  and  Larher  [Larher],  as  well  as  2D  calculations  by 
Sander  [Sander]. 

2  Gelb  and  Muller’s  Method 

Gelb  and  Muller’s  method  consists  of  two  parts:  the  molecular  dynamics  simulation 
using  the  temperature  quench  method  and  the  analysis  of  the  simulation  results. 

2.1  Temperature  Quench  Simulations 

Figure  2.1  illustrates  the  temperature  quench  method  for  the  molecular  dynamics 
simulations.  The  system  starts  in  a  single-phase  equilibrated  state,  shown  as  point 
A.  The  temperature  is  then  suddenly  dropped  to  a  point  B  at  a  lower  temperature. 
This  places  the  system  in  an  unstable  state  and  causes  a  spontaneous  separation 
into  two  co-existing  phases.  At  this  point,  it  is  not  necessary  for  the  simulation  to 
continue  until  the  system  is  once  again  in  equilibrium.  Only  local  equilibrium  is 
required. 

2.2  Analysis  of  simulation  results 

Analysis  of  a  snapshot  of  the  system  after  the  above  temperature  quench  simulation 
gives  two  points  on  the  phase  co-existence  curve  at  the  final  temperature.  One 
will  be  for  the  lower  density  phase  (vapour  in  the  example)  and  one  for  the  higher 
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Figure  2.1:  Example  phase  co-existence  curve  with  temperature  in  the  vertical  di¬ 
rection  and  density  in  the  horizontal  direction.  The  temperature  is  dropped  from  an 
equilibrated  initial  condition  at  A  to  a  lower  temperature  at  B,  causing  a  separation 
of  phases.  Figure  based  on  [1]  Fig.  1. 


density  phase  (liquid  in  the  example).  The  lower  portion  of  the  phase  diagram 
can  be  constructed  in  this  way,  however  the  method  is  not  successful  in  the  broad, 
flat  region  of  the  phase  diagram.  In  this  region  there  is  a  range  of  densities  for 
one  temperature  at  which  the  phases  co-exist  and  thus  it  is  difficult  to  identify  the 
phases. 

Gelb  and  Muller  analysed  a  snapshot  from  the  simulation  by  first  dividing  the 
simulation  box  into  sub- cel  Is.  The  density  of  each  of  these  sub- cells  was  then  calcu¬ 
lated,  as  well  as  the  local  co-ordination  number  of  each  particle.  Boundary  particles 
were  identified  as  those  particles  that  had  a  local  co-ordination  number  greater  than 
the  low-density  phase  and  less  than  the  high-density  phase.  Sub-cells  that  consisted 
of  more  than  30%  boundary  particles  were  excluded  from  the  analysis. 

A  histogram  of  the  remaining  cell  densities  was  constructed.  Examples  of  such 
histograms  for  our  krypton  on  graphite  system  are  shown  in  Figure  2.2.  The  his¬ 
togram  of  densities  separates  into  a  bi-modal  distribution,  reflecting  the  densities 
of  the  phases  that  co-exist  in  the  system.  The  density  of  each  of  these  phases  is 
identified  as  the  most  common  density  within  the  range  of  densities  for  that  phase. 

As  can  be  seen  from  Figure  2.2(b),  the  histogram  can  have  more  than  one  promi¬ 
nent  peak  in  the  range  of  densities  tliat  correspond  to  one  of  the  co-existing  phases. 
It  is  the  highest  peak  which  is  taken  to  be  the  relevant  density.  Further,  for  high  (or 
low)  total  densities,  there  may  not  be  enough  sub-cells  with  densities  in  the  range  of 
densities  of  the  other  phase  for  a  confident  assessment  of  the  peak  in  the  histogram. 
This  is  illustrated  in  Figure  2.2(c). 

The  results  obtained  by  Gelb  and  Muller’s  method  of  analysis  are  shown  in 
Figure  2.3.  For  comparison,  results  of  molecular  dynamic  simulations  with  10  000 
atoms  (instead  of  the  2496  particles  used  here)  are  indicated,  as  well  as  experimental 
results  and  numerical  2D  results.  The  method  was  repeated  for  total  densities  of 
0.467  and  0.612. 
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Figure  2.2:  (a)  Histogram  of  densities  including  all  sub-cells  (dashed  green  line)  and 
including  only  sub-cells  with  less  than  30%  of  particles  being  boundary  particles 
(solid  blue  line).  Temperature=75K,  total  density=0.467.  (b)  As  for  (a)  but  area 
under  solid  blue  line  normalised  to  1.  Teinperature=80K,  total  density=0.467.  (c) 
As  for  (b).  Temperature=80K,  total  density=0.612. 


3  Pattern  Boundary  Detection  Method 

Instead  of  following  Gelb  and  Muller’s  method  of  excluding  sub-cells  with  too  many 
boundary  particles  and  constructing  histograms  of  densities,  in  the  pattern  boundary 
detection  method  we  use  the  boundary  particles  to  define  the  fluid  regions.  The  areas 
of  such  fluid  regions  are  then  calculated  by  summing  the  area  of  the  Voronoi  cells 
of  the  constituent  particles.  This  yields  a  density  for  the  fluid  region  that  does  not 
exclude  any  particles  in  its  calculation.  It  also  allows  us  to  get  over-  and  under¬ 
estimates  of  the  fluid  density  by  accounting  for  the  boundary  particles  in  different 
ways. 

The  major  differences  from  Gelb  and  Muller’s  method  are  thus: 

1.  Find  boundary  particles  by  counting  nearest  neighbours  and  nearest  neigh¬ 
bours  of  nearest  neighbour  particles. 

2.  Calculate  area  of  each  phase  by  finding  the  Voronoi  cell  of  each  particle  and 
not  by  using  areas  of  sub-cells. 

3.  Include  all  particles  and  regions  and  do  not  arbitrarily  eliminate  regions. 

4.  Find  over  and  under  estimates  for  points  on  phase  co-existence  curve  by  at¬ 
tributing  different  areas  to  fluid  regions. 
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KryiXorVGfapNta  Phase  Diagram 


Figure  2.3:  Comparison  of  phase  co-existence  points  found  by  Gelb  and  Muller’s 
method  of  analysis  with  molecular  dynamics,  experimental  and  2D  calculation  re¬ 
sults. 


Figure  3.1  shows  a  typical  snapshot  from  a  simulation  of  2496  particles  at  a 
final  temperature  of  70K  with  a  total  density  of  0.467.  The  highlighted  particles 
are  within  the  simulation  box.  This  box  has  periodic  boundary  conditions  in  both 
directions.  In  order  to  see  the  neighbouring  particles  and  calculate  the  Voronoi  cells 
on  the  edges,  the  particles  have  been  repeated  outside  the  simulation  box.  Fluid 
particles  are  coloured  blue,  boundary  particles  are  coloured  green,  and  solid  particles 
are  coloured  red. 

In  the  solid  phase,  particles  are  in  a  triangular  lattice  configuration  with  6  nearest 
neighbours.  Fluid  particles  have  less  than  6  nearest  neighbours.  Boundary  particles 
have  less  than  6  nearest  neighbours  and  have  at  least  one  nearest  neighbour  that 
has  6  nearest  neighbours.  Thus,  by  counting  the  number  of  nearest  neighbours 
of  each  particle  and  keeping  track  of  which  neighbours  have  fewer  than  6  nearest 
neighbours,  each  particle  can  be  identified  as  either  a  fluid  particle,  a  solid  particle, 
or  a  boundary  particle. 

3.1  Calculating  Densities 

Having  identified  the  type  of  each  particle,  the  density  of  the  fluid  regions  can  be 
calculated  in  the  following  way.  Let  Ux  denote  the  number  of  particles  of  type  x  and 
let  Ax  denote  the  sum  of  the  area  of  all  Voronoi  cells  corresponding  to  particles  of 
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Figure  3.1:  Typical  snapshot  from  simulations,  showing  Voronoi  cells  of  partieles  in 
doubly  periodic  box.  Particles  within  the  siiiuilation  box  are  highlighted;  blue  = 
fluid  particle,  green  =  boundary  particle,  red  =  solid  particle. 


type  X.  The  fluid  density  is  then  given  by 

Fluid  Density  = - ^ 

{Afluid  Aixiry)  ~  y^bdry  ^  y 

The  area  in  the  denominator  is  the  area  attributed  to  the  fluid  partieles.  The  first 
term  corresponds  to  the  area  of  the  Voronoi  cells  coloured  yellow  in  Figure  3.1.  The 
second  term  is  an  estimate  for  the  area  occupied  by  the  boundary  partieles.  The 
Voronoi  cells  of  the  boundary  particles  can  not  be  used  to  calculate  the  area  occupied 
because  the  boundary  particle  is  to  be  treated  as  the  edge  of  the  solid  phase.  The 
boundary  particles  are  each  attributed  an  area  equal  to  the  average  area  oecupied 
by  a  particle  in  the  solid  phase. 

An  under-estimate  for  the  fluid  density  does  not  take  into  aeeount  the  area 
occupied  by  the  boundary  particles  and  thus  over-estimates  the  area  occupied  by 
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the  fluid  region: 

Fluid  Density  Under  —  Estimate  =  — — - .  (3.2) 

A  fluid  "f" 

An  over-estimate  for  the  fluid  density  can  be  obtained  by  using  a  different  def¬ 
inition  of  fluid  particles.  Designate  a  particle  as  a  large  cell  particle  if  its  Voronoi 
cell  has  an  area  greater  than  1.5  times  the  average  area  of  the  solid  particle  cells. 
Then 

Fluid  Density  Over  —  Estimate  =  (3  3) 

‘^large  cells 

Figure  3.2  shows  the  particles  that  are  designated  as  large  cell  particles  in  black, 
with  their  Voronoi  cells  coloured  yellow. 


Figure  3.2:  Particles  defined  to  be  large  cell  particles  are  coloured  black  for  the  same 
simulation  as  shown  in  Figure  3.1.  The  Voronoi  cell  of  such  particles  are  coloured 
yellow. 


To  calculate  the  density  of  the  solid  ph£Lse,  only  particles  in  the  inner  core  of 
the  solid  regions  were  used.  Such  inner  core  particles  have  the  correct  number  of 
nearest  neighbours  and  each  of  their  nearest  neighbours  also  has  the  correct  number 
of  nearest  neighbours.  The  inner  core  of  the  solid  region  is  less  likely  to  have  defects 
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than  locations  close  to  the  edge  of  the  solid  region.  The  density  is  given  by: 

Solid  Density  =  duster  _  ^3  4^ 

^solid  inner  cluster 

The  inner  core  regions  can  be  seen  in  Figure  3.3  for  the  same  simulation  as  shown 
in  Figure  3.1. 


Figure  3.3:  Particles  in  the  inner  core  of  the  solid  regions  are  coloured  magenta,  for 
the  same  simulation  as  shown  in  Figure  3.1. 


4  Results 

The  points  obtained  on  the  phase  co-existence  curve  for  krypton  on  graphite  using 
the  Pattern  Boundary  Detection  method  are  shown  in  Figure  4.1.  The  molecular 
dynamics  simulation,  experimental  and  2D  calculation  results  are  shown  for  com¬ 
parison.  The  different  colours  (yellow,  magenta  and  cyan)  refer  to  different  total 
densities  that  were  used  for  the  simulations.  The  small  coloured  dots  on  the  left 
side  of  the  phase  diagram  correspond  to  the  under-  and  over-estimates  of  the  fluid 
density. 
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It  can  be  seen  that  the  points  on  the  left  of  the  phase  coexistence  curve  agree 
well  with  the  molecular  dynamics  simulation  results  at  low  temperatures.  As  the 
temperature  increases  and  the  phase  coexistence  curve  becomes  flatter,  the  method 
performs  worse,  as  expected.  The  results  for  the  right  side  of  the  pheise  diagram 
(density  of  the  solid  phaise)  have  an  unexplained  constant  offset  from  the  molecular 
dynamics  simulation  results.  Note  that  simulations  were  carried  out  at  each  tem¬ 
perature  for  each  of  the  total  densities  -  coloured  points  that  can  not  be  seen  in  the 
figure  are  obscured  by  other  points,  indicating  a  good  agreement  between  results 
from  different  total  densities. 

4.1  Speed-up 

The  main  advantage  of  the  Pattern  Boundary"  Detection  Method  is  that  it  is  faster 
than  a  long  molecular  dynamics  simulation  that  requires  the  system  to  be  in  equi¬ 
librium.  The  temperature  quench  method  needs  only  one  equilibrated  single-phase 
initial  condition.  This  can  then  be  used  to  obtain  results  at  a  variety  of  lower 
temperatures.  After  the  temperature  is  dropped  suddenly,  only  local  equilibrium  is 
necessary  and  so  there  is  no  need  to  wait  for  the  whole  system  to  equilibrate. 

The  Pattern  Boundary  Detection  method  yields  two  points  on  the  pheise  diagram 
from  one  final  temperature,  thus  halving  the  number  of  simulations  that  need  to  be 
carried  out.  Furthermore,  identifying  the  phases  requires  only  one  snapshot  from 
the  simulation.  There  is  no  global  quantity,  such  as  an  order  parameter,  that  needs 
to  be  traeJeed  over  successive  snapshots  (as  for  example  in  methods  that  observe  the 
heat  capacity). 

A  further  advantage  of  the  Pattern  Boundary  Detection  Method  is  that  it  pro¬ 
vides  under-  and  over-estimates  of  the  fluid  density.  Up  until  the  broad,  flat  region 
of  the  phase  diagram  (where  the  method  is  not  expected  to  work  anyway),  these 
estimates  bounded  the  results  that  were  obtained  by  using  any  of  the  total  densities 
considered. 

4.2  Extension  to  the  neck  region  of  the  phase  diagram 

The  method,  as  described  above,  identifies  the  fluid  and  solid  phases  by  counting 
nearest  neighbours.  This  is  effectively  a  measure  of  the  local  density.  At  higher  total 
densities,  the  difference  in  density  between  the  fluid  and  solid  phases  is  no  longer 
large  enough  for  this  approach  to  be  successful.  These  high  total  density  simulations 
must  be  considered  so  as  to  extend  the  Pattern  Boundary  Detection  method  to  the 
neck  region  of  the  phase  diagram  (in  the  upper  right  of  Figure  4.1). 

The  distinguishing  featiure  of  the  solid  phase  in  these  situations  is  the  order  of  the 
particles.  They  form  a  triangular  lattice  by  adsorption  of  atoms  onto  the  graphite 
substrate.  Thus  a  measure  of  the  order,  or  geometry,  of  the  particles  must  be  used, 
such  as  the  Defect  Me€isure.  This  is  ongoing  work. 

A  further  complication  in  the  neck  region  of  the  phase  diagram  is  that  the 
temperature  is  high  enough  such  that  particles  that  have  been  adsorbed  onto  the 
substrate  move  significantly,  while  remaining  adsorbed.  There  is  a  spiral  motion 
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about  the  adsorption  site.  To  identify  the  solid  particles,  this  motion  about  the 
adsorption  site  must  be  averaged  out  by  looking  at  a  number  of  snapshots. 

5  Conclusions 

The  Pattern  Boundary  Detection  method  provides  a  way  of  constructing  the  lower 
portion  of  the  phase  diagram  of  krypton  on  graphite.  Speed-up  over  equilibrated 
molecular  dynamics  simulations  comes  from  using  the  temperature  quench  method 
in  simulations,  from  finding  two  points  on  the  phase  co-existence  curve  for  every 
final  temperature,  and  from  needing  only  one  snapshot  per  final  temperature. 
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Figure  4.1:  Phase  diagram  for  krypton  on  graphite  showing  the  results  of  the  Pattern 
Boundary  Detection  method  (in  yellow,  magenta  and  cyan),  as  well  as  molecular 
dynamics,  experimental  and  2D  calculation  results.  Small  coloured  dots  indicate 
under-  and  over-estimates  of  the  fluid  density. 
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1  Introduction 

The  structure  of  crystalline  solids  has  been  an  interest  of  the  condensed  matter 
and  materials  science  fields  for  a  long  time.  The  defects  present  in  these  solids 
determine  many  of  the  physical  properties  of  the  material.  Investigations  have  been 
both  theoretical  and  experimental,  concentrating  on  how  different  types  of  defects, 
and  the  number  of  them,  affect  mechanical,  electrical  and  optical  properties.  A  large 
number  of  experiments  have  been  performed  that  attempt  to  make  materials  that 
are  free  of  defects,  or  examine  the  formation  energies  and  movement  of  defects.  The 
number  of  defects  and  their  type  is  commonly  deduced  from  bulk  properties  of  the 
material,  such  as  diffraction  patterns  or  shear  stress.  The  focus  is  on  the  material 
as  a  whole  rather  than  on  the  environment  of  each  constituent  particle. 

This  report  is  concerned  with  a  quantitative  assessment  of  the  quality  of  two- 
dimensional  lattices.  We  are  interested  not  in  the  way  that  a  lattice  responds  to  some 
external  stimulus  that  would  measure  a  bulk  property  of  the  lattice  as  a  whole,  but 
rather  assessing  the  positions  of  the  constituent  particles  in  a  manner  similar  to  the 
human  eye.  The  aim  is  to  quantify  what  the  eye  sees  when  comparing  two  lattices 
and  deciding  that  one  is  better  than  the  other.  The  notion  of  ’’better”  may  depend 
on  which  property  of  the  lattice  is  more  important  to  the  assessor,  or  the  goal  of 
assessing  the  lattice.  The  measures  of  the  defects  of  a  lattice  that  we  have  developed 
is  thus  concerned  with  the  local  neighbourhood  of  each  particle,  reflecting  the  eye’s 
propensity  to  judge  sub-regions  of  the  lattice  and  how  these  regions  combine. 

Four  defect  metrics  that  have  this  local  nature  are  described  and  compared.  The 
most  versatile  of  these,  the  Defect  Measure,  is  used  as  a  tool  in  applications  that 
arise  from  the  challenge  of  designing  an  isotropic  potential  that  leads  to  the  self- 
assembly  of  particles  into  a  lattice.  All  particles  are  identical  and  move  in  a  finite 
two  dimensional  area. 

The  self-assembly  of  particles  is  of  importance  in  the  diverse  fields  of  under¬ 
standing  how  biological  or  chemical  components  form  a  coherent  whole  and  multi- 
vehicular  surveillance.  Large  numbers  of  small  vehicles  moving  in  a  lattice  formation 
has  been  proposed  as  an  efficient  way  of  surveying  the  landscape.  (Ref?)  The  lo¬ 
cal  deviations  from  a  perfect  lattice  formation  must  be  understood  in  this  context. 
Vehicles  that  communicate  only  with  their  neighbouring  vehicles  should  be  able 
to  assemble  into  and  maintain  a  lattice  formation,  eliminating  the  need  for  each 
vehicle’s  specific  trajectory  to  be  programmed.  The  design  of  the  ideal  isotropic 
potential  for  doing  so  requires  an  effective  measure  of  the  defects  in  the  lattice,  as 
does  the  evaluation  of  the  robustness  of  that  potential. 

2  Types  of  Lattice  Defects 

There  are  a  number  of  different  types  of  defects  that  can  occur  in  a  monatomic 
two-dimensional  lattice.  The  defects  most  commonly  referred  to  are  listed  below. 

•  Vacancy:  a  lattice  site  that  should  have  a  particle  is  unoccupied. 
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•  Interstitial:  a  particle  occupies  a  lattice  site  that  should  not  have  a  particle 
occupying  it. 

•  Frenkel  pair:  a  vacancy  and  an  interstitial  are  nearby.  A  particle  is  at  a  lattice 
site  which  should  be  unoccupied,  leaving  a  nearby  lattice  site  that  should  be 
occupied  empty. 

•  Topological  defect:  a  region  in  a  lattice  where  the  ordered  structure  is  different 
to  the  rest  of  the  lattice.  For  example,  in  a  honeycomb  lattice  a  region  that 
has  five  particles  in  a  ring  rather  than  the  required  six. 

•  Split  interstitial:  two  particles  share  a  lattice  site,  typically  by  having  their 
centre  of  mass  at  the  lattice  site  where  there  should  be  one  particle. 

•  Edge  dislocation:  an  extra  line  of  particles  inserted  part  of  the  way  into  the 
lattice.  The  adjacent  lines  of  correctly  ordered  particles  bend  around  the  line 
that  terminates.  Dislocations  are  breaks  in  the  translational  symmetry  of  the 
lattice. 

•  Disclinations:  a  line  defect  that  results  in  a  rotation  if  the  orientation  of  the 
lattice  around  the  defect  is  tracked. 

•  Grain  boundaries:  regions,  typically  lines,  where  the  orientation  of  the  lattice 
changes  abruptly.  Frequently  caused  by  two  lattices  growing  separately  and 
then  meeting. 


3  Metrics  for  Assessing  the  Quality  of  Lattices 

VV^e  describe  four  defect  metrics  that  focus  on  the  local  configuration  of  particles. 
These  metrics  are  compared  in  the  following  section. 

3.1  Defect  Measure 

The  Defect  Measure  is  a  tool  that  was  developed  in  order  to  compare  the  quality  of 
lattices.  The  human  eye  is  frequently  a  good  judge  of  the  quality  of  a  lattice,  how¬ 
ever,  a  more  qualitative  assessment  was  sought  in  order  to  efficiently  assess  lattices 
that  are  formed  during  the  optimization  procedure  used  for  finding  a  potential  that 
leads  to  the  self-assembly  of  particles  in  a  plane. 

Given  particle  positions  in  a  plane,  the  quality  of  the  lattice  that  is  formed  is 
determined  by  the  desired  or  target  lattice.  If  the  target  lattice  is  known  then 
calculating  the  Defect  Measure  requires  only  Step  3  in  the  procedure  described 
below. 

If  the  target  lattice  is  not  known,  or  if  it  is  necessary  to  hnd  the  type  of  lattice 
that  the  system  of  particles  is  forming,  then  Steps  1  and  2  can  be  applied,  to  deter¬ 
mine  what  the  target  lattice  is.  Identification  of  a  target  lattice  means  identifying 
both  the  type  of  lattice  (honeycomb,  triangular,  etc.)  and  the  minimum  lattice 
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spacing,  which  we  shall  call  the  lattice  constant,  a.  That  is,  identifying  the  target 
lattice  involves  identifying  the  shape  and  the  scaling. 

The  need  to  identify  the  target  lattice  arises  when  optimizing  the  potential  for  the 
self-assembly  of  a  honeycomb  lattice.  The  competing  lattice  is  a  triangular  lattice 
that  may  have  one  of  two  different  lattice  constants,  depending  on  the  density  of 
particles.  When  searching  over  parameters  for  the  honeycomb  potential,  if  it  can 
be  seen  that  the  lattice  that  is  forming  for  a  set  of  parameters  is  heading  towards 
a  triangular  lattice  rather  than  a  honeycomb  lattice,  the  simulation  can  be  stopped 
and  another  set  of  parameters  may  be  tried.  In  this  way,  the  optimization  procedure 
can  be  sped  up. 

Identifying  the  lattice  constant  is  important  not  only  because  it  contributes  to 
the  identification  of  the  target  lattice  but  also  because  for  some  potentials,  the  lattice 
that  is  formed  has  a  different  lattice  constant  than  that  suggested  by  the  isotropic 
inter-particle  potential  that  led  to  the  formation  of  the  lattice.  For  example,  a 
potential  designed  to  form  a  honeycomb  lattice  with  a  lattice  constant  that  is  equal 
to  the  distance  to  the  first  minimum  of  the  potential,  may  form  a  lattice  with  a 
different  lattice  constant  due  to  the  sensitivity  to  the  density  of  particles.  In  assessing 
the  quality  of  the  resulting  lattice,  if  only  the  shape  of  the  lattice  is  important,  then 
the  actual  lattice  constant  must  be  found. 

The  algorithm  for  identifying  the  target  lattice  and  computing  the  Defect  Mea¬ 
sure  does  not  need  the  positions  of  all  particles  in  the  plane.  It  only  requires  a  list  of 
distances  to  the  nearest  20  neighbours  of  each  particle.  To  distinguish  between  the 
four  types  of  lattices  considered  -  triangular,  honeycomb,  square  and  Kagome,  it  is 
only  necessary  to  consider  the  distances  to  the  nearest  15  neighbours  of  each  parti¬ 
cle,  however  results  are  more  reliable  (identification  of  target  lattice  is  improved)  if 
more  nearest  neighbours  are  considered. 

This  is  an  important  feature  for  our  surveillance  example.  Vehicles  would  only 
need  to  detect  other  vehicles  that  are  in  a  certain  range  that  covers  an  area  in  w^hich 
there  would  be  approximately  20  other  vehicles.  The  direction  of  each  detected 
vehicle  is  unimportant,  only  the  distance  to  that  vehicle.  In  this  way,  the  position 
of  each  vehicle  does  not  need  to  be  tracked. 

In  the  identification  steps  (Steps  1  and  2),  it  is  assumed  that  the  lattice  is 
reasonably  well  formed.  This  assumption  effectively  means  that  the  human  eye 
would  be  able  to  distinguish  the  type  of  target  lattice. 

Any  number  of  particles  may  be  in  the  lattice  being  assessed,  if  periodic  boundary 
conditions  are  imposed.  If  this  is  not  the  case,  then  the  number  of  particles  must 
be  large  enough  such  that  the  number  of  inner  particles  of  the  lattice  out-number 
the  number  of  boundary  particles.  The  larger  the  number  of  inner  particles  with 
respect  to  boundary  particles,  the  better  the  algorithm  is  able  to  identify  the  target 
lattice. 
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3.1.1  Procedure  for  identifying  the  target  lattice  and  computing  the 
Defect  Measure 

Step  1:  Identify  lattice  constantf  a 

1.  List  inter-particle  distances  dpj  for  particle  p  in  ascending  order. 

2.  Find  clusters  of  values  for  each  particle. 

3.  Average  over  values  in  clusters  of  lowest  values  to  find  a. 


Step  2:  Identify  type  of  lattice 

1.  Count  how  many  values  in  first  few  clusters  identified  in  Step  1. 

2.  Find  mode  of  number  of  particles  in  clusters  of  lowest  values,  then  clusters  of 
next  lowest  values,  etc  (for  a  majority  of  particles,  the  number  of  values  in 
each  cluster  should  be  the  same). 

3.  Compare  number  of  particles  at  each  distance  (modes  of  clusters)  with  known 
values  for  possible  types  of  lattices. 

Number  of  particles  in  perfect  lattice: 


Mode  of: 

cluster  1 

cluster  2 

cluster  3 

Triangular 

6 

6 

6 

Square 

4 

4 

4 

Honeycomb 

3 

6 

3 

Kagome 

4 

4 

6 

Step  3:  Compute  the  Defect  Measure 

Compare  the  given  lattice  to  a  perfect  lattice  of  the  same  type,  with  the  same 
lattice  constant,  to  find  a  measure  of  the  defects,  i.e.  the  quality  of  the  lattice. 

For  a  particle  p  in  the  target  lattice,  find  the  distance  r  that  is  halfway  between 
the  distance  to  the  closest  neighbours  and  the  next  closest  neighbours.  This  distance 
is  shown  as  the  red  circle  in  Figure  3.1.  For  a  triangular  lattice,  there  are  6  closest 
neighbours  at  a  distance  of  a  and  6  next  closest  neighbours  at  a  distance  of  \/(3)a. 
The  red  circle  has  a  radius  of  r  =  (1  +  y/(3))a/2. 

Define  the  nearest  neighbours  of  a  particle  p  to  be  those  particles  that  are  within 
a  radius  r  of  particle  p. 

1.  Choose  weights,  oJdefect  type  for  ^a-ch  type  of  defect  (see  discussion  below). 

2.  For  each  particle,  p,  construct  the  nearest  neighbours  circle  and  compute  the 
Defect  Measure  of  that  particle  according  to  which  of  the  following  types  of 
defects  apply  (shown  in  Figure  3.2): 
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Figure  3.1:  The  nearest  neighbours  circle  (red  circle)  of  particle  p  is  halfway  between 
the  closest  particles  and  the  next  closest  particles. 


•  Displaced  particles 


(Defect  Measure)p  =  (^displaced  x 


\r 


f  fjp  X  {dpj  a)^ 

jGnearest  neighbours 


•  Missing  particles 

(Defect  Measure)p  =  x  njjiissijig  x  a 

•  Extra  particles 


(Defect  Mcasure)p  =  UJextra  X  llextra  X  a^ 


•  Lone  particles 

(Defect  Measure)p  =  (^missing  x  6  x  +  w/one  x 

•  Boundary  particles 


(Defect  MeaSUre)p  =  Ldb<mndary  X 

3.  The  Defect  Measure  for  the  lattice  is  given  by  summing  over  all  particles  p: 

(Defect  Measure  for  Lattice)  =  ^(Defect  Measure)p 

p 

ffjp  is  the  fade  factor  for  particle  j  with  respect  to  particle  p.  The  fade  factor 
allows  particles  to  faxle  out  of  view  of  particle  p  rather  than  disappear  as  they  cross 
the  nearest  neighbours  circle.  The  fade  factor  is  equal  to  1  for  dpj  <  (5  +  3v/3)a/8 
and  is  equal  to  0  for  dpj  >(!-{-  \/3)a/2  (the  nearest  neighbours  circle).  The  fade 
factor  decreases  from  1  to  0  over  a  distance  that  is  equal  to  a  quarter  of  the  distance 
from  a  to  the  nearest  neighbours  circle.  Within  this  region,  the  fade  factor  decreases 
in  a  cubic  polynomial  fashion  with  horizontal  tangent  at  the  end  points  of  the  region. 
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(a)  Displaced  particles  (b)  Missing  particles  (c)  Extra  particles 


(d)  Lone  particles 


(e)  Boundary  particles 


Figure  3.2:  Type  of  defects  used  in  computing  the  Defect  Measure 


The  weights,  oJde/ect  type^  for  each  type  of  defect  are  chosen  according  to  the 
severity  of  the  defect.  This  depends  on  the  goal.  For  example,  in  the  surveil¬ 
lance  situation,  if  the  collision  avoidance  of  the  vehicles  is  an  issue,  then  a  larger 
weight  would  be  given  to  the  weight  for  extra  particles,  cOextrai  hi  order  to  deter 
more  strongly  lattices  with  extra  particles.  When  optimizing  the  potential  for  the 
self-assembly  problem,  a  larger  weight  for  boundary  particles,  cobaundaryf  n^ay  be 
necessary  to  penalize  the  formation  of  distinct  sub-lattices.  Note  that  boundary 
particles  are  not  penalized  for  missing  particles  in  their  nearest  neighbours  circle. 
The  effect  of  changing  the  weights  for  the  various  types  of  defects  will  be  discussed 
further  in  Section  4. 

One  of  the  advantages  of  the  Defect  Measure  is  that  it  allows  this  flexibility  to 
penalize  different  types  of  defects  more  heavily.  In  this  way,  it  is  a  tool  that  can  be 
shaped  for  the  specific  task  at  hand. 

The  Defect  Measure  is  lower  for  higher  quality  lattices.  A  perfect  lattice  will 
only  have  a  Defect  Measure  equal  to  zero  if  the  weight  for  boundary  particles  is  set 
to  zero. 

Note  that  all  of  the  varieties  of  defects  discussed  in  Section  2  are  taken  into 
account  by  the  types  of  defects  listed  in  Step  3.2.  For  example,  grain  boundaries  are 
taken  into  account  by  contributions  to  the  Defect  Measure  from  displaced,  missing 
and  extra  particles. 

The  Defect  Measure  essentially  compares  the  local  density  around  each  particle 
to  that  of  the  target  lattice,  with  a  correction  for  the  distance  between  particles, 
added  on. 

For  this  reason,  only  the  inter- particle  distances  are  required.  The  geometry  of 
the  particles  is  not  considered.  This  is  sufficient  because,  for  any  potential  that  could 
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leaxi  to  self-assembly,  if  the  correct  number  of  particles  are  put  into  a  region  the  size 
of  the  nearest  neighbours  circle,  they  will  arrange  themselves  into  approximately 
the  correct  configuration  due  to  the  potential  between  them, 

3.1.2  Advantages  of  the  Defect  Measure 

The  Defect  Measure  has  a  number  of  advantages  over  the  methods  for  quantifying 
the  quality  of  lattices  surveyed  at  the  beginning  of  Section  3. 

•  The  Defect  Measure  gives  a  local  assessment  of  the  quality  of  a  lattice.  Apart 
from  the  versatility  of  the  Defect  Measure  in  applications  that  this  leads  to, 
a  local  assessment  of  a  lattice  is  closer  to  the  qualitative  assessment  that  a 
human  eye  would  make  of  a  lattice. 

•  Each  particle’s  contribution  to  the  quality  of  the  lattice  can  be  quantified.  In 
this  way,  regions  of  the  lattice  that  are  not  well  formed  can  be  identified.  This 
is  useful  in  applications  such  as  the  one  discussed  in  Section  5.4. 

•  The  primary  types  of  defects,  and  the  number  of  such  defects,  that  occur  in  a 
lattice  can  be  easily  identified. 

•  The  flexibility  of  the  Defect  Measure  due  to  the  assignment  of  weights  to 
defects,  leads  to  a  versatility  that  may  be  exploited  in  applications, 

•  The  target  lattice  does  not  need  to  be  specified. 

•  The  Defect  Measure  is  invariant  under  rotations,  reflections  and  translations 
of  the  lattice. 

3.1.3  Identifying  the  boundary  of  a  lattice 

To  implement  the  procedure  for  calculating  the  Defect  Measure  (with  non-zero 
weights  for  the  boundary  particles),  the  particles  that  form  the  boundary  of  the 
lattice  must  be  identified,  using  only  the  distances  to  the  nearest  20  neighbours. 
The  boundary  particles  of  a  perfect  lattice  can  be  identified  by  counting  the 
number  of  particles  at  a  distance  of  a,  and  the  number  of  particles  within  a  distance 
of  2a  from  each  particle.  For  a  triangular  lattice,  an  inner  particle  has  6  neighbours 
at  a  distance  of  a,  6  neighbours  at  a  distance  of  \/3a,  and  6  neighbours  at  a  distance 
of  2a.  Boundary  particles  can  have  a  maximum  of  5  particles  at  a  distance  of  a 
and  15  particles  within  a  distance  of  2a.  This  is  shown  in  Figure  3.3.  Particles 
that  satisfy  these  conditions  are  designated  as  boundary  particles.  Note  that  these 
requirements  specify  the  maximum  concavity  of  the  boundary  that  can  be  detected. 
A  similar  construction  applies  to  different  types  of  lattices. 

For  an  imperfect  triangular  lattice,  the  condition  of  having  a  maximum  of  5 
particles  at  a  distance  of  a  is  loosened  to  having  a  maximum  of  5  particles  within 
the  nearest  neighbours  circle.  The  second  condition  is  relaxed  to  having  a  maximum 
of  15  particles  within  a  distance  (1  4-  \/7/2)a  of  the  candidate  boundary  particle. 
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Figure  3.3:  A  boundary  particle  in  a  triangular  lattice  can  have  a  maximum  of  5 
particles  at  a  distance  of  a  and  15  particles  within  a  distance  of  2a.  The  red  particle 
shows  such  a  particle.  The  red  line  indicates  the  boundary  of  the  lattice.  The  circle 
encloses  particles  that  are  within  a  distance  2a  of  the  red  particle. 


This  distance  is  halfway  between  2a  and  the  distance  to  the  next  nearest  neighbours 
(at  \/7a).  Similarly  relaxed  conditions  apply  to  other  types  of  lattices. 

Identifying  the  boundary  of  a  lattice  given  the  positions  of  particles 

If  a  system  of  particles  is  not  in  a  reasonably  well-formed  lattice,  yet  the  goal  is 
to  compute  the  Defect  Measure  of  the  particles  with  respect  to  some  specified  target 
lattice,  then  it  will  be  necessary  to  use  a  different  method  to  identify  the  boundary 
particles  (for  non-periodic  boundary  conditions).  In  this  situation,  the  position  of 
each  particle  is  required,  as  the  number  of  particles  in  the  circles  described  above 
may  not  be  at  all  comparable  to  that  of  the  target  lattice. 

The  convex  hull  of  a  set  of  points  in  a  plane  is  the  minimal  convex  set  containing 
all  the  points.  It  may  be  visualised  as  the  shape  of  an  elastic  band  that  has  been 
stretched  to  encompass  all  the  points  and  then  allowed  to  collapse  around  them. 
The  convex  hull  for  a  set  of  points  and  the  points  identified  as  boundary  points  by 
this  convex  hull  are  shown  in  Figure  3.4. 

Clearly,  this  is  not  what  the  human  eye  identifies  as  the  boundary.  It  is  the 
non-convex  hull  that  correctly  identifies  the  boundary  particles.  First,  a  minimum 
concave  curvature,  p,  must  be  chosen.  The  non-convex  vertices  of  the  boundary  are 
then  those  particles  that  are  touched  by  a  disk  of  radius  p  as  it  is  rolled  around  the 
outside  of  the  set  of  points.  The  following  algorithm  for  finding  the  non-convex  hull 
is  due  to  E.  Boje  [Boje  [2000]]. 

Algorithm  for  finding  the  non-convex  hull  (Boje): 

1.  Find  the  Delaunay  triangulation*  of  the  set  of  points. 

2.  Find  the  outside  triangles,  i.e.  those  triangles  with  an  edge  that  does  not 
touch  another  triangle’s  edge.  Such  edges  together  form  the  convex  hull. 

'The  Delaunay  triangulation  of  a  set  of  points  is  a  triangulation  such  that  no  point  is  inside  the 
circumcircle  of  any  triangle  in  the  triangulation.  It  is  the  dual  graph  of  the  Voronoi  tessellation  of 
the  points. 
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(a)  Convex  hull 


(b)  Boundary  points 


Figure  3.4:  (a)  The  convex  hull  of  a  set  of  points  in  the  plane,  (b)  The  points 
identified  by  the  convex  hull  as  boundary  points  (shown  in  red). 


3.  Recursively  delete  any  outside  triangle  that  has  an  outside  edge  longer  than 

2p. 

4.  Recursively  delete  outside  triangles  whose  outside  edge  is  the  longest  and 
whose  circumscribing  circle^  has  a  radius  greater  than  p. 

5.  Iterate  until  all  triangles  pass  steps  3  and  4. 

The  points  identified  by  this  algorithm  as  the  particles  on  the  non-convex  hull  of 
the  set  of  points  in  Figure  3.4  is  shown  in  Figure  3.5  in  red.  These  are  the  boundary 
particles.  Note  that  the  boundary  points  identified  by  the  convex  hull  corresponds 
to  the  points  found  by  rolling  a  disk  of  infinite  radius  around  the  set  of  points. 


Figure  3.5:  The  points  identified  by  the  non-convex  hull  as  boundary  points  (shown 
in  red). 


^The  circumcircle  of  a  polygon  is  a  circle  that  paiis«>  through  all  of  the  vertices  of  the  polygon. 
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3.2  Geometric  Defect  Measure 

The  Defect  Measure  takes  into  account  the  local  density  of  particles  and  the  distance 
between  particles.  It  does  not  consider  the  local  geometry  of  particles.  Despite  this, 
as  discussed  above,  the  Defect  Measure  provides  a  good  assessment  of  the  quality 
of  a  lattice.  The  Geometric  Defect  Measure  was  developed  as  an  alternative  quality 
assessment  tool  that  could  be  compared  to  the  Defect  Measure  to  check  that  looking 
at  local  densities  (with  a  correction  for  displacements)  does  indeed  lead  to  a  lattice 
with  the  correct  geometry.  It  can  also  be  combined  with  the  Defect  Measure  as  a 
correction,  to  yield  a  quality  assessment  tool  that  considers  local  densities,  distances 
between  particles  and  local  geometry. 

Computation  of  the  Geometric  Defect  Measure  requires  the  position  of  each 
particle  in  the  system  as  well  as  the  type  of  target  lattice.  It  focuses  on  the  shape 
of  the  lattice  and  not  the  scaling.  Thus  two  lattices  that  differ  only  by  a  scaling  of 
the  lattice  constant  will  have  identical  values  for  the  Geometric  Defect  Measure.  It 
is  computationally  more  expensive  than  the  calculation  of  the  Defect  Measure. 


A 


•  m2  • 


Figure  3.6:  Construction  used  in  computing  the  Geometric  Defect  Measure  for  par¬ 
ticle  p. 

The  algorithm  for  computing  the  Geometric  Defect  Measure  of  a  honeycomb 
lattice  is  outlined  below  and  illustrated  in  Figure  3.6.  The  procedure  is  similar  for 
other  types  of  lattices. 

Algorithm  for  computing  the  Geometric  Defect  Measure  of  a  honeycomb  lattice: 

1.  Find  the  nearest  neighbour  of  particle  p.  Label  it  nnl.  Let  the  distance 
between  particles  p  and  nnl  be  dnni- 

2.  Extend  the  line  from  p  to  nnl  a  distance  of  dnni-  Determine  whether  there 
is  a  particle  within  a  distance  dnni/8  of  this  point,  i.e.  whether  there  is  a 
particle  in  region  2  in  Figure  3.6. 

3.  Continue  extending  the  line  from  p  to  nnl  in  units  of  dnni  and  determining 
whether  a  particle  is  within  a  region  around  the  end  points.  Do  this  for  a  total 
of  5  regions  or  steps  from  particle  p. 
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4.  Compare  whether  or  not  a  particle  is  found  in  each  region  to  that  expected 
from  a  perfect  lattice.  For  the  honeycomb  lattice,  there  should  be  particles  in 
regions  3  and  4,  but  not  in  regions  2  and  5.  (Region  1  will  have  a  particle  by 
construction.) 

5.  If  a  region  j  does  not  have  the  correct  number  of  particles  then  ripij  =  1,  else 

ripij  =  0. 

6.  Repeat  for  the  second  and  third  nearest  neighbours,  nn2  and  nn3. 

7.  Calculate  the  angle  between  pairs  of  nearest  neighbours  of  p:  Opu,  ^pi3,  and 

^p23- 

8.  Sum  over  all  particles  p  in  the  lattice  to  obtain  the  Geometric  Defect  Measure 
of  the  lattice. 

3/5  3 

Geometric  Defect  Measure  =  EE  E  npij  +  E  I  cos  120®  —  cos  0pik\ 

p  1=1  \j=l  k^2,k>i 

(3.1) 

If  the  system  of  particles  does  not  have  periodic  boundary  conditions  then  the 
extension  of  the  lines  from  particles  to  their  nearest  neighbours  should  be  cut  off 
when  a  boundary  is  reached. 

A  number  of  modifications  to  the  algorithm  outlined  above  are  appropriate  for 
most  lattices.  Firstly,  the  number  of  steps  that  the  lines  are  extended  may  be 
increased  or  decreased,  depending  on  the  type  of  lattice.  For  the  triangular  lattice, 
looking  at  only  3  regions  is  sufficient  to  give  a  quality  assessment  that  is  comparable 
to  what  the  human  eye  would  judge.  However,  5  regions  is  more  appropriate  for  a 
honeycomb  lattice.  Changing  the  number  of  steps  taken  alters  how  local  the  quality 
assessment  is.  It  is  the  local  nature  of  the  Geometric  Defect  Measure  that  makes  it 
useful  for  detecting  regions  with  many  defects. 

Secondly,  the  size  of  the  regions  used  to  determine  whether  a  particle  is  in  the 
correct  position  relative  to  the  base  particle  p  may  be  adjusted.  It  is  appropriate 
to  increase  the  size  of  the  detection  region  the  further  the  region  is  from  the  base 
particle.  Doing  so  is  compatible  with  judgements  made  by  the  human  eye.  It  is 
also  affected  by  the  importance  of  having  a  correctly  aligned  lattice  rather  than  a 
skewed  lattice.  For  the  honeycomb  lattice,  a  good  choice  is  allowing  the  radius  of 
the  detection  region  for  region  s  to  be  (s  —  1)  x  dnni/8  for  s  >  1. 

Lastly,  as  regions  further  away  from  the  base  particle  are  less  important  in 
quantifying  the  local  geometry,  the  contribution  of  the  more  distant  regions  to  the 
Geometric  Defect  Measure  can  be  reduced.  The  first  term  in  parentheses  in  Equa¬ 
tion  3.1  then  becomes  ripij  x  (6  —  s)/4,  for  s  >  1  where  s  denotes  the  number 
of  the  detection  region. 

3.3  Voronoi  Metric 

The  Voronoi  Metric  finds  the  Voronoi  tessellation  of  the  particles  in  the  lattice  and 
compares  the  area  of  each  Voronoi  cell  to  the  area  of  a  Voronoi  cell  of  the  target 
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lattice.  The  Voronoi  tessellation  of  a  set  of  particles  in  the  plane  is  the  partition  of 
the  plane  into  regions  such  that  any  point  of  the  plane  in  the  region  corresponding 
to  particle  p  is  closer  to  p  than  to  any  other  particle.  The  Voronoi  tessellation  of  an 
imperfect  lattice  can  be  seen  in  Figure  3.7(a). 

The  Voronoi  Metric  is  straightforward  to  apply  to  lattices  with  periodic  bound¬ 
ary  conditions.  The  positions  of  the  particles  near  the  bounding  box  are  mapped 
outside  the  bounding  box  in  an  appropriate  way  that  is  consistent  with  the  periodic 
boundary  conditions.  This  is  illustrated  in  Figure  3.7(b). 

The  particles  of  the  lattice  are  shown  in  blue;  the  lattice  has  a  bounding  box 
specified  by  0  <  x  <  1  and  0  <y  <  1.  The  particles  that  have  been  mapped  outside 
the  bounding  box  are  coloured  cyan.  The  particle  on  the  left  that  is  coloured 
magenta  lies  inside  the  bounding  box.  It  is  mapped  to  the  particle  position  on 
the  right  that  is  coloured  red.  Such  a  construction  allows  the  Voronoi  cells  of  the 
particles  close  to  the  boundary  to  be  calculated  without  edge  effects,  for  periodic 
boundary  conditions. 

Algorithm  for  computing  the  Voronoi  Metric: 


1.  Map  particles  that  are  close  to  the  bounding  box  outside  the  bounding  box, 
respecting  the  periodic  boundary  conditions. 

2.  Find  a  Voronoi  tessellation  of  the  particles. 

3.  Compute  the  area  of  each  Voronoi  cell  that  contains  a  particle  in  the  original 
lattice. 


4.  The  Voronoi  metric  is  given  by 

N 

Voronoi  Metric  =  ^ 


area(V(p))  — 


area  of  bounding  box 


N 


where  N  is  the  number  of  particles  in  the  original  lattice  and  V(p) 
Voronoi  cell  containing  particle  p. 


(3.2) 
is  the 


An  alternative  expression  for  the  area  of  a  Voronoi  cell  in  the  perfect  lattice  may 
be  obtained  from  the  geometry  of  the  target  lattice  and  the  best  estimate  for  the 
lattice  constant.  The  lattice  constant  may  be  estimated  by  Step  1  of  the  procedure 
for  computing  the  Defect  Measure  described  in  Section  3.1.1.  This  is  particularly 
important  for  the  honeycomb  lattice,  for  which  the  lattice  constant  depends  not  only 
on  the  first  minima  of  the  potential  but  also  the  density  of  particles.  For  a  perfect 
honeycomb  lattice  (with  periodic  boundary  conditions),  the  area  of  each  Voronoi 
cell  would  be  3\/3a^/4.  The  second  term  in  Equation  3.2  may  be  replaced  by  this 
expression. 

For  lattices  formed  in  a  bounding  box  that  does  not  have  periodic  boundary 
conditions,  an  assessment  of  the  quality  of  the  lattice  based  on  the  Voronoi  Metric 
can  be  made  by  ignoring  the  contribution  from  boundary  particles.  The  boundary 
particles  can  be  identified  by  using  Boje’s  algorithm  for  finding  the  non-convex  hull 
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(a) 


(b) 

Figure  3.7:  (a)  Voronoi  tessellation  of  a  set  of  p'oints  in  the  plane,  (b)  The  same 
Voronoi  tessellation  as  in  (a)  with  the  partieles  of  the  original  lattiee  eoloured  blue 
and  the  partieles  that  are  mapped  outside  the  periodic  boundaries  coloured  cyan. 
The  particle  on  the  left  coloured  magenta  is  mapped  outside  the  bounding  box  to 
the  particle  position  coloured  red  on  the  right. 


of  a  set  of  points.  In  this  case,  the  alternative  expression  for  the  area  of  a  Voronoi 
cell  in  the  target  lattice,  explained  in  the  preceding  paragraph,  should  be  used. 

To  include  a  contribution  from  the  boundary  particles  of  the  lattice,  there  are  a 
number  of  options.  Modihed  Voronoi  cells  for  the  boundary  particles  can  be  formed 
by  taking  the  area  enclosed  by  the  lines  of  the  Voronoi  tessellation  and  the  bounding 
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box.  These  areas  for  the  boundary  particles  can  then  be  compared  to  the  areas  of  the 
similarly  modified  Voronoi  cells  that  would  be  formed  for  the  target  lattice.  Since 
the  modified  Voronoi  cells  of  the  boundary  particles  in  the  target  lattice  depend  a 
great  deal  on  the  construction  of  the  target  lattice,  it  is  best  to  sum  the  areas  of  the 
modified  Voronoi  boundary  cells  of  the  lattice  and  compare  this  to  the  sum  of  the 
areas  of  the  modified  boundary  cells  of  the  target  lattice,  rather  than  compare  the 
areas  of  individual  cells.  However,  this  is  not  an  effective  way  to  assess  the  quality 
of  a  lattice  because  of  the  many  possible  ways  of  constructing  the  target  lattice. 

Another  way  to  include  the  contribution  of  the  boundary  particles  to  a  Voronoi- 
like  metric  is  to  define  the  Voronoi  cells  of  the  boundary  particles  to  be  the  area 
enclosed  by  the  lines  of  the  Voronoi  tessellation  and  the  non-convex  hull  of  the 
particles.  This  method  suffers  from  the  same  drawback  as  the  previous  one,  but  has 
the  advantage  that  for  a  lattice  that  is  not  aligned  with  the  edges  of  the  bounding 
box,  the  contribution  of  the  boundary  particles  to  the  value  of  the  metric  will  be 
smaller.  This  effect  is  important  for  lattices  that  do  not  require  the  pressure  from 
the  domain  walls  to  form.  For  example,  this  method  is  the  appropriate  way  to 
include  the  boundary  particles  in  the  Voronoi  metric  of  a  triangular  lattice  that  is 
formed  in  a  domain  that  is  larger  than  the  area  occupied  by  the  lattice. 

The  boundary  particles  can  also  be  dealt  with  in  the  following  manner.  As¬ 
sign  to  each  boundary  particle  of  the  lattice  the  area  of  an  inner  Voronoi  cell  in 
the  target  lattice.  For  example,  each  boundary  particle  of  a  honeycomb  lattice 
formed  in  a  domain  without  periodic  boundary  conditions,  would  be  assigned  an 
area  of  3\/3a^/4.  The  area  occupied  by  the  cells  of  the  boundary  particles  is  then 
Afxiry  =  ’^Wry3\/3a^/4,  where  n^ry  is  the  number  of  boundary  particles.  Compute 
the  area,  of  the  domain  that  is  occupied  by  the  inner  particles  of  the  lat¬ 

tice.  The  contribution  of  the  boundary  particles  to  the  Voronoi-like  metric  is  then 
l^bdry  ”  -^tnncrl* 

None  of  these  options  for  including  the  contribution  of  the  boundary  particles 
of  the  lattice  to  a  Voronoi-like  metric  are  satisfactory,  for  the  reasons  mentioned 
above  as  well  as  the  variability  at  the  edges  of  computed  Voronoi  tessellations.  The 
Voronoi  Metric  is  thus  of  limited  use  for  lattices  formed  in  domains  without  periodic 
boundary  conditions. 

3.4  Cumulative  Distribution  Function  Metric 

The  cumulative  distribution  function  of  the  inter-particle  distances  of  a  lattice  can 
be  used  to  assess  the  quality  of  the  lattice.  This  metric  was  proposed  by  Mezic  and 
Runolfsson  in  a  different  setting  [  Mezic  and  Runolfsson  [2004]].  The  cumulative 
distribution  function  (CDF)  metric  is  most  effective  when  only  inter-particle  dis¬ 
tances  up  to  a  distance  of  slightly  above  2a  are  considered.  In  a  perfect  lattice,  this 
includes  the  nearest  neighbours,  the  next  nearest  neighbours  and  the  third  nearest 
neighbours  of  each  particle. 

Algorithm  for  computing  the  Cumulative  Distribution  Function  Metric: 

1.  Let  the  distance  to  a  point  half  way  between  the  third  nearest  neighbours 
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circle  and  fourth  nearest  neighbours  circle  of  particles  in  the  target  lattice  be 
(Unax- 


2.  For  each  distance  d  from  d  =  0  to  d  =  dmax^  all  inter- particle  distances  of 
the  lattice  being  assessed  that  are  less  than  or  equal  to  d. 


3.  Then 


CDF, add)  =  (3.3) 

dtj<d,  i>j 


for  0  <  d  <  djnaxi  where  the  dij  are  the  inter-particle  distances  of  the  lattice. 

4.  CDF  target  Is  defined  in  a  similar  way  using  the  inter-particle  distances  of  the 
target  lattice. 


5.  The  Cumulative  Distribution  Function  Metric  is  given  by 


\CDFiadl)-CDFu.rgcdl)\dl 


(3.4) 


The  CDF  of  a  lattice  and  its  target  lattice  is  shown  in  Figure  3.8.  CDFiat  is  in 
blue  and  CDF  target  is  in  green.  The  value  of  the  Cumulative  Distribution  Function 
Metric  is  the  area  between  the  curves. 


Figure  3.8:  The  Cumulative  Distribution  Function  Metric  computes  the  area  be¬ 
tween  CDFiat  shown  in  blue  and  CDF  target  shown  in  green. 


The  question  of  how  to  treat  the  boundary  particles  of  a  lattice  is  also  an  issue 
for  this  metric,  especially  for  domains  without  periodic  boundary  conditions.  For 
lattices  formed  in  domains  with  periodic  boundary  conditions,  the  only  concern  is 
whether  the  domain  can  indeed  be  filled  with  a  perfect  lattice  with  the  specified 
number  of  particles.  Not  all  domains  have  a  perfect  lattice  that  completely  fills  the 
domain  for  an  arbitrary  number  of  particles.  However,  this  introduces  only  a  very 
minor  error  into  the  value  of  the  CDF  metric. 

For  domains  without  periodic  boundary  conditions,  a  target  lattice  must  be 
constructed  that  can  be  used  to  find  CDFtargeu  The  inter-particle  distances  of  the 
target  lattice  depend  on  where  the  boundary  particles  are  placed,  i.e.  the  shape  of 
the  boundary. 
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A  more  serious  problem  with  the  CDF  metric  is  that  there  is  a  cancellation 
between  two  different  types  of  defects:  missing  particles  and  extra  particles.  If 
a  particle  is  missing  a  neighbouring  particle  at  a  distance  d'  from  it  and  another 
particle  has  an  extra  particle  at  a  distance  d!  from  it,  then  there  will  be  some 
cancellation  between  these  two  defects,  and  the  value  of  the  CDF  metric  at  d!  will 
be  lower  than  it  should  be.  However,  these  two  defects  also  affect  the  surrounding 
particles  and  this  will  add  to  the  value  of  the  CDF  metric.  How  much  is  added 
depends  on  the  arrangement  of  particles  around  the  defect  and  not  the  defects 
themselves. 

The  CDF  metric  was  designed  to  only  look  at  inter- particle  distances  less  than 
(Unax  partially  for  this  reason.  There  is  less  opportunity  for  such  cancellation  of 
defects  to  occur.  Another  reason  for  limiting  the  CDF  metric’s  horizon  to  dmax  is 
that  considering  all  inter- particle  distances  would  put  too  much  emphasis  on  the 
long  range  order  of  the  lattice.  When  assessing  the  quality  of  a  lattice,  the  human 
eye  tends  to  focus  more  on  the  order  within  regions  that  have  a  radius  of  a  few 
lattice  constants,  rather  than  the  long  range  order  of  the  lattice  as  a  whole. 

A  feature  of  the  CDF  metric  is  that  it  tends  to  judge  lattices  with  grain  bound¬ 
aries  relatively  harshly.  This  may  or  may  not  be  a  concern  depending  on  the  goal 
and  how  severe  such  a  defect  is  considered  to  be. 

4  Comparison  of  Quality  Metrics 

A  natural  question  to  ask  is:  Which  of  the  metrics  for  assessing  the  quality  of  a 
lattice  discussed  in  Section  3  is  the  best? 

Evidently,  this  depends  on  which  properties  of  a  lattice  are  more  important. 
Different  metrics  focus  on  different  aspects,  such  as  having  the  correct  number  of 
particles  in  approximately  the  right  positions,  or  having  the  correct  alignment  of 
particles.  This  will  be  divscussed  further  below. 

There  are  two  straightforward  ways  to  compare  metrics  that  assess  the  quality 
of  a  lattice:  whether  it  can  identify  the  best  lattice  from  a  set  of  lattices,  and  the 
computational  time  taken  to  compute  the  value  of  the  metric  for  a  lattice. 

In  order  to  compare  the  metrics  from  Section  3,  a  set  of  20  lattices  were  gener¬ 
ated,  with  576  particles  in  a  domain  with  periodic  boundary  conditions.  The  target 
lattice  was  the  honeycomb  lattice.  Each  metric  was  used  to  assess  the  lattices  and 
rank  them  from  best  to  worst.  The  time  taken  to  calculate  the  value  of  the  metric 
for  each  lattice  was  averaged  over  the  20  lattices.  The  results  are  shown  in  Figure  4.1 
and  Table  4. 

Figure  4.1  has  the  metrics  discussed  in  Section  3  along  the  vertical  axis  and  the 
number  of  each  of  the  20  lattices  used  for  the  comparison  along  the  horizontal  axis. 
Defect  Measure  1  and  Defect  Measure  2  differ  only  in  the  weights  assigned  to  the 
different  types  of  defects.  ^  The  colours  in  the  figure  represent  the  ranking  of  the  20 
lattices,  with  a  rank  of  1  in  red  being  the  best  lattice  and  a  rank  of  20  in  dark  blue 

W61ghtS  for  Defect  Messure  1:  ^displaced  ~  l-OiCVmi««in^  “  “  0,8. 

Weights  for  Defect  Measure  2:  ufdtAptarrd  =  10,u;mi55in.g  =  0.02,  =0.015. 

There  were  no  boundary  or  lone  particles  in  these  lattices. 
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Rank 


D«fKt  M«a5ure ) 

Defect  Measure  2 


Geometric  Defect  Measure 


Voronol  Metric 

CDf  Metric 


Figure  4.1:  Comparion  of  quality  assessment  metrics.  Each  metric  along  the  vertical 
axis  ranked  the  20  test  lattices  (along  horizontal  axis).  The  colon rbar  indicates  which 
colour  corresponds  to  which  ranking.  Red  signifies  the  best  lattice  (rank  1)  and  dark 
blue  the  worst  lattice  (rank  20). 


being  the  worst  lattice.  For  example,  along  the  top  row  it  can  be  seen  that  Defect 
Measure  1  ranked  lattice  number  7  as  the  best  lattice  and  lattice  number  10  as  the 
w’orst  lattice. 


Time  to  compute  (s) 

Defect  Measure 

0.117 

Geometric  Defect  Measure 

6.69 

Voronoi  Metric 

0.338 

CDF  Metric 

0.163 

Figure  1.1  shows  that  Defect  Measure  1  ranks  the  test  lattices  in  an  order  that 
is  similar  to  the  ranking  of  the  Voronoi  Metric  (with  the  notable  exception  of  the 
identification  of  the  best  lattice).  Rows  2  and  3  of  Figure  4.1  show  that  Defect 
Measure  2  assigns  similar  rankings  to  the  test  lattices  as  the  Geometric  Defect 
Measure.  This  highlights  the  flexibility  of  the  Defect  Measure  which  results  from 
the  freedom  to  choose  the  weights  for  the  defects.  It  will  be  shown  below  that 
the  Geometric  Defect  Measure  and  the  Voronoi  Metric  consider  different  aspects  of 
lattices  to  be  important  and  thus  apply  to  different  situations.  The  CDF  Metric 
identifies  the  same  few  lattices  as  being  the  worst  lattices  that  the  other  metrics 
identify.  However,  the  lattices  that  are  judged  to  be  the  best  lattices  by  the  CDF 
Metric  are  not  judged  to  be  that  way  by  the  other  metrics.  The  Geometric  Defect 
Measure  ranks  the  CDF  Metric’s  best  lattices  as  being  only  moderately  good. 

Lattices  number  18  and  5  are  shown  in  Figure  4.2.  It  can  be  seen  that  in  lattice 
18,  although  there  are  defects  like  missing  and  extra  particles,  the  particles  tend  to 
be  aligned  with  each  other.  For  a  majority  of  particles,  the  angles  between  nearest 
neighbour  particles  are  close  to  that  of  a  perfect  honeycomb  lattice.  There  also 
seems  to  be  more  medium  range  structure  than  in  lattice  5.  This  is  precisely  what 
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(a)  Lattice  18  (b)  Lattice  5 

Figure  4.2:  Test  honeycomb  lattices  with  576  particles  and  periodic  boundary  con¬ 
ditions.  The  dark  blue  particles  are  inside  the  bounding  box.  The  cyan  particle 
positions  show  the  structure  at  the  edge  of  the  bounding  box. 


the  Geometric  Defect  Measure  focuses  on.  However,  in  lattice  18,  there  are  obvious 
defects.  In  lattice  5  on  the  other  hand,  the  density  of  particles  is  much  more  uniform 
across  the  domain. The  particles  are  not  aligned  well  into  a  honeycomb  lattice  but  a 
majority  of  particles  have  the  correct  number  of  nearest  neighbours  and  form  rough 
rings  of  6  particles.  It  is  this  local  area  of  each  particle  aspect  of  lattices  that  is  the 
focus  of  the  Voronoi  Metric.  The  shape  of  the  Voronoi  cells  is  not  considered,  only 
their  area  compared  to  a  Voronoi  cell  of  a  perfect  lattice. 

Which  of  lattices  18  and  5  is  judged  to  be  the  better  lattice  depends  on  the 
goal.  If  the  purpose  of  having  a  honeycomb  lattice  is  to  cover  an  area  evenly,  with 
each  particle  having  3  nearest  neighbours  and  forming  rings  of  6  particles,  then  the 
Voronoi  Metric  is  the  one  to  use.  If  the  goal  is  to  form  as  much  of  a  close- to- per  feet 
honeycomb  lattice  as  jx>ssiblc  then  the  Geometric  Defect  Measure  should  be  used.  ^ 
It  is  interesting  to  note  that  the  Defect  Measure  can  be  used  to  achieve  both  of  these 
goals  simply  by  adjusting  the  weights  for  the  different  types  of  defects. 

Both  the  Voronoi  Metric  and  the  Geometric  Defect  Measure  ranked  lattice  18 
as  the  best  lattice.  The  Geometric  Defect  Measure  did  so  because  of  the  regular 
alignment  of  the  particles.  The  Voronoi  Metric  ranked  it  so  highly  because  of  the 
peculiar  coincidence  of  the  error  in  the  area  of  the  Voronoi  cells  around  the  gross 
defects  that  can  be  seen  in  lattice  18,  summing  to  a  similar  error  in  area  that  is 
spread  out  across  all  Voronoi  cells.  This  can  be  seen  by  comparing  the  lattices  in 
Figure  4.2;  lattice  18  was  ranked  as  the  best  and  lattice  5  was  ranked  as  the  second 
best.  It  is  because  of  a  few  large  errors  in  area  summing  to  a  similar  total  as  the  sum 
of  many  small  errors  in  area,  and  not  because  the  Voronoi  Metric  particularly  looks 
at  geometry,  that  lattice  18  has  the  best  ranking.  It  can  be  seen  from  Figure  4.1 
that  Defect  Measure  1  avoids  this  anomaly. 

Lattice  18  was  generated  using  a  polynomial  potential  while  lattice  5  was  generated  using  a 
Rechtsman  style  potential  Reclitsman,  Stillinger,  and  Torquato  [2006] 
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The  Voronoi  Metric  whose  results  are  shown  in  Figure  4.1  used  the  alternative 
expression  for  the  area  of  a  Voronoi  cell  in  the  perfect  lattice  that  is  explained  in 
Section  3.3.  The  alternative  expression  computes  the  average  latttice  constant  and 
then  sets  the  perfect  Voronoi  cell  area  to  be  3>/3a^/4,  based  on  geometry.  The 
assessment  of  this  quality  metric  thus  depends  greatly  on  how  accurate  the  estimate 
of  the  lattice  constant  is.  The  average  lattice  constant  is  calculated  using  Step  1  of 
the  algorithm  for  computing  the  Defect  Measure. 

The  Voronoi  Metric  can  also  be  implemented  as  in  the  algorithm  in  Section  3.3, 
with  the  area  of  a  perfect  Voronoi  cell  being  given  by  (areaof  bounding  box) /(number  of  particles). 
The  results  for  this  metric  are  shown  in  Figure  4.3  under  the  label  Voronoi  Metric  2. 

Lattice  18  is  no  longer  ranked  highly,  while  most  of  the  other  rankings  remain  the 
same.  Using  this  version  of  the  metric  concentrates  more  on  how  much  of  the  avail¬ 
able  space  a  particle’s  Voronoi  cell  covers  rather  than  comparing  it  to  the  space  a 
particle  in  a  perfect  lattice  would  cover. 


Rank 

Defect  Measure  1 

Defect  Measure  2 

Nearest  Neighbour 
Lines  Measure 

Angle  Measure 
Geometric  Defect  Measure 
Voronoi  Metric  1 
VOronoi  Metric  2 
CDF  Metric 
Area  Estimate  Measure 

;  «  a  >  ID  17  14  H  <■  HI 

Lattice  number 


Figure  4.3:  Comparison  of  quality  assessment  metrics.  Each  metric  along  the  vertical 
axis  ranked  the  20  test  lattices  (along  horizontal  axis).  The  colourbar  indicates  which 
colour  corresponds  to  which  ranking.  Red  signifies  the  best  lattice  (rank  1)  and  dark 
blue  the  worst  lattice  (rank  20). 


Figure  4.3  also  shows  the  two  components  of  the  Geometric  Defect  Measure:  one 
focusing  on  whether  particles  are  where  they  should  be  along  the  nearest  neighbour 
lines  and  one  focusing  on  how  close  the  angles  between  nearest  neighbours  are  to 
what  they  should  be  in  a  perfect  lattice.  The  rankings  for  these  two  components 
are  very  similar. 

The  last  row  of  Figure  4.3,  labeled  Area  Estimate  Measure,  is  a  very  rough 
quality  assessment  metric.  It  compares  the  area  a  perfect  lattice  would  cover  if  it 
had  the  average  lattice  constant,  to  the  area  of  the  bounding  box.  That  is, 


Area  Estimate  Measure  = 


I  Area  of  bounding  box  —  (3\/3a^/4)  x  (number  of  particles)  | . 


This  quick  calculation  can  identify  the  best  lattice  and  the  worst  lattice  but  does 
not  perform  well  in  between.  It’s  only  advantage  is  that  it  requires  nothing  more 
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than  the  calculation  of  the  average  lattice  constant.  All  of  the  other  metrics,  require 
this  computation  and  then  other  computations. 

Table  4  shows  the  time  taken  to  compute  the  value  of  each  metric.  These  times 
were  obtained  by  averaging  the  time  taken  to  compute  the  metric  for  each  of  the  20 
test  lattices.  Each  metric  was  given  the  minimum  information  it  needed  in  order  to 
compute  the  value  of  the  metric.  The  Defect  Measure  calculations  were  given  the 
distances  to  the  nearest  twenty  neighbours  of  each  particle.  The  CDF  Metric  was 
given  a  list  of  all  nearest  neighbour  distances  up  to  a  distance  of  2.2a.  The  Geometric 
Defect  Measure  and  the  Voronoi  Metric  were  given  the  particle  positions.  The 
rationale  for  giving  each  quality  metric  only  the  minimum  information  that  it  needs 
stems  from  the  applications  of  the  metrics.  If  the  Defect  Measure  is  used  to  assess 
the  lattice  formed  by  vehicles  flying  in  formation  then  these  vehicles  need  only  detect 
their  nearest  twenty  neighbours  -  this  limits  the  range  necessary  for  their  relative 
distance  sensors  and  the  amount  of  information  that  must  be  transmitted.  When 
using  the  Defect  Measure  to  assess  the  quality  of  lattices  formed  in  a  LA  MM  PS 
simulation,  the  distance  to  the  nearest  twenty  neighbours  is  easily  accessible  due 
to  the  structure  of  the  LAMMPS  simulation  code.  This  code  calculates  particle 
positions  in  parallel  by  dividing  up  the  domain  into  smaller  regions,  thus  keeping 
more  detailed  information  about  a  particle’s  nearest  neighbours.  Similar  reasoning 
holds  for  the  CDF  Metric. 


(a)  Lattice  1  (b)  Lattice  10 

Figure  4.4:  The  two  worst  test  honeycomb  lattices  with  576  particles  and  periodic 
boundary  conditions.  The  dark  blue  particles  arc  inside  the  bounding  box.  The 
cyan  particle  positions  show  the  structure  at  the  edge  of  the  bounding  box. 


Figure  1.1  shows  that  all  of  the  metrics,  apart  from  the  CDF  Metric,  found 
lattices  1  and  10  to  be  the  two  worst  lattices.  These  lattices  are  shown  in  Figure  4.4. 
It  can  be  seen  that  these  two  worst  lattices  also  exhibit  the  two  different  types  of 
lattice  that  the  best  lattices  in  Figure  4.2  did.  Namely,  one  has  particles  that  are  well 
aligned  (albeit  in  the  wrong  locations)  and  the  other  has  a  more  uniform  number  of 
particles  per  area  (though  not  aligned  into  the  honeycomb  pattern  at  all). 
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(a)  Lattice  15  (b)  Lattice  16 

Figure  4.5:  The  CDF  Metric’s  two  best  test  honeycomb  lattices  with  576  particles 
and  periodic  boundary  conditions.  The  dark  blue  particles  are  inside  the  bounding 
box.  The  cyan  particle  positions  show  the  structure  at  the  edge  of  the  bounding 
box. 


The  CDF  Metric  assigns  rankings  that  are  quite  different  from  all  of  the  other 
metrics.  It  judges  the  worst  lattices,  similarly  to  the  other  metrics  but  chooses 
different  lattices  as  the  best  lattices.  These  best  lattices,  shown  in  Figure  4.5, 
received  a  moderately  good  ranking  from  the  Geometric  Defect  Measure  but  were 
rated  as  quite  bad  lattices  by  the  Voronoi  Metric,  One  feature  that  they  exhibit 
is  having  regions  that  are  well- formed  that  are  separated  from  other  well- formed 
regions  by  regions  with  many  defects.  Lattices  15  and  16  have  large  regions  with 
missing  particles  and  also  curves  with  a  small  distance  between  the  particles  (more 
evident  in  Lattice  16).  This  combination  leads  to  some  cancellation,  causing  the 
lattice  to  be  ranked  highly,  as  discussed  in  Section  3.4. 

Thus,  it  seems  that  the  CDF  Metric  does  not  appear  to  assign  rankings  that 
are  similar  to  what  a  human  observer  would  assign,  whether  particle  alignment  or 
density  is  the  focus.  The  CDF  Metric  is  an  indicator  of  how  many  inter-particle 
distances  (within  a  limited  range  set  by  dmax)  a-re  correct.  Since  the  distances  are 
the  focus  rather  than  the  local  density  or  alignment  (properties  that  are  important 
for  forming  a  lattice  structure),  the  structure  is  less  important  with  this  metric.  This 
renders  the  CDF  Metric  less  useful  as  a  quality  metric  for  the  particular  problem  of 
the  self-assembly  of  particles  into  a  target  lattice. 

5  Applications  of  the  Defect  Measure 

The  Defect  Measure  is  a  tool  that  can  be  used  to  quantitatively  assess  the  quality 
of  lattices.  It  is  useful  in  a  variety  of  situations,  such  as  finding  potentials  that  lead 
to  the  self-assembly  of  particles,  assessing  the  robustness  of  such  potentials,  and 
helping  to  speed  up  simulations  of  lattice  formation. 
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5.1  Design  of  Potential  for  Self-Assembly 

The  Defect  Measure  can  be  used  as  a  tool  not  only  to  assess  the  quality  of  lattices 
that  result  from  simulations  with  a  particular  isotropic  potential  between  particles, 
but  also  to  refine  such  potentials  to  achieve  the  best  potential.  For  example,  the 
isotropic  inter- particle  potential  necessary  for  the  self-assembly  of  a  honeycomb 
lattice  will  have  a  strongly  repulsive  component  at  short  inter-particle  distances, 
a  local  minimum  at  the  lattice  constant  of  the  target  lattice,  a  global  minimum  at 
\/3  times  the  lattice  constant,  and  a  tail  that  goes  to  zero  at  long  inter-particle 
distances.  A  functional  form  for  such  a  jx)tential  can  be  designed  with  a  number  of 
parameters  that  may  be  varied  to  achieve  the  potential  that  yields  the  best  lattices, 
as  measured  by  the  Defect  Measure. 

Rechtsman  and  co-workers  [Rechtsman,  Still inger,  and  Torquato  [2006]]  found 
the  following  expression  for  the  self-assembly  of  particles  into  a  honeycomb  lattice: 

~  ^  ^  exp[-a2r)  -  0.4  exp[-40(r  -  03)^]  (5. 1 ) 

with  ao  =  5.89,  ai  =  17.9,  02  =  2.49,  and  03  =  1.823  as  the  best  parameter  values  for 
good  lattices.  The  form  of  Equation  5.1  can  be  used  to  refine  the  potential  and  find 
the  best  value  of  parameter  02  for  self-assembly.  The  other  parameter  values  and 
conditions  (such  as  the  density  and  cooling  schedule)  were  kept  constant  in  the  1000 
simulations  that  were  run  with  65  particles  in  a  domain  without  periodic  boundary 
conditions.  The  parameter  02  values  used  for  the  1000  simulations,  formed  a  Gaus¬ 
sian  distribution  with  mean  02  =  1.49  and  standard  deviation  of  0.6.  Figure  5.1 
shows  the  Defect  Measure  of  the  final  lattice  of  each  simulation  versus  the  value  of 
the  02  parameter  used  to  generate  the  lattice.  The  values  of  the  weights  for  the 
Defect  Measure  are:  (^displaced  ~  1*0,  UJmissing  ~  ^extra  ~  0.8,  (^boundary  ”  0.2, 
and  uiifync  =  2.0.  These  weights  are  the  same  weights  as  for  Defect  Measure  1  in 
Section  4.  The  lattices  corresponding  to  the  red  and  green  points  are  shown  in  Fig¬ 
ure  5.2.  The  green  point  has  the  lowest  Defect  Measure  of  all  the  lattices.  Note  that 
each  point  in  Figure  5.1  is  the  result  of  one  simulation  with  random  initial  velocities 
for  the  particles.  Ideally,  for  each  value  of  02  a  number  of  simulations  would  be  run 
and  the  average  Defect  Measure  of  the  final  lattices  used. 

Figure  5. 1  indicates  that  for  the  conditions  under  which  the  simulations  were  run, 
the  best  choice  for  parameter  02  is  a  value  of  2.6.  Note  that  the  honeycomb  lattice  is 
quite  fragile  and  particularly  sensitive  to  the  density  of  particles  and  the  boundary 
conditions.  Thus  for  different  densities  or  boundary  conditions,  other  values  of  02 
may  be  more  appropriate.  However,  the  method  for  designing  an  isotropic  potential 
has  been  illustrated  by  the  above  example. 

As  mentioned  in  Section  4,  the  weights  of  the  Defect  Measure  should  be  chosen 
according  to  the  properties  of  the  lattice  that  are  most  important. 

The  Geometric  Defect  Measure  can  also  be  used  to  design  the  potential  for  the 
formation  of  lattices  with  the  correct  alignment  of  particles.  Figure  5.3(a)  shows 
the  Geometric  Defect  Measure  versus  the  parameter  02  for  the  same  simulations  as 
above.  The  best  lattice,  shown  in  Figure  5.3(b),  was  formed  with  02  =  2.23.  Thus 
which  value  of  02  is  chosen  depends  on  whether  the  focus  is  the  correct  alignment  or 
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Figure  5.1;  Defect  Measure  versus  paxameter  a2  in  Equation  5.1.  The  1000  simula¬ 
tions  shown  here  used  65  particles  in  a  domain  without  periodic  boundary  conditions. 
The  02  values  have  a  mean  of  1.49  and  a  standard  deviation  of  0.6.  The  lattices 
corresponding  to  the  red  and  green  points  are  shown  in  Figure  5.2. 
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Figure  5.2:  (a)  Lattice  corresponding  to  the  green  point  in  Figure  5.1,  with  02  =  2.6 
and  Defect  Measure=  9.6.  (b)  Lattice  corresponding  to  the  red  point  in  Figure  5.1, 
with  02  =  0.5  and  Defect  Measure^  41.6. 


local  density  of  particles.  Figure  5.4  depicts  the  two  components  of  the  Geometric 
Defect  Measure:  the  Nearest  Neighbour  Lines  Measure  and  the  Angle  Measure, 

5.2  Optimization  of  Potential  for  Self-Assembly 

When  employing  an  optimization  procedure  to  search  over  parameter  space  in  order 
to  find  the  best  potential  for  the  self-assembly  of  particles,  the  Defect  Measure 
can  be  a  useful  tool  in  a  number  of  ways.  Firstly,  it  quantitatively  measures  the 
quality  of  each  lattice,  thus  removing  the  need  for  the  user  to  visually  assess  each 
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(a)  (b) 

Figure  5.3:  (a)  Geometric  Defect  Measure  versus  parameter  a2.  (b)  Lattice  corre¬ 
sponding  to  the  lowest  Geometric  Defect  Measure  in  (a),  with  a2  =  2.23. 


lattice.  Secondly,  the  procedure  for  computing  the  Defect  Measure,  explained  in 
Section  3.1,  can  identify  the  type  of  lattice  (if  it  is  reasonably  well-formed).  This 
is  particularly  useful  when  the  target  lattice  is  the  honeycomb  lattice  because  the 
competitor  lattice  is  the  very  stable  triangular  lattice.  In  the  parameter  space  of 
the  honeycomb  self-assembly  potential,  the  Defect  Measure  may  have  local  minima 
corresponding  to  triangular  lattices.  Step  2  of  the  Defect  Measure  algorithm  can 
identify  such  local  minima  and  the  optimization  procedure  can  be  constructed  so  as 
to  avoid  parameters  leading  to  triangular  lattices. 

The  third  use  of  the  Defect  Measure  in  an  optimization  scheme  is  indicating 
when  the  simulations  can  be  ended.  For  each  set  of  parameters,  a  simulation  will 
have  to  be  run  to  determine  whether  those  parameters  lead  to  a  good  potential  for 
self-assembly.  Rather  than  having  to  run  each  large  simulation  for  a  long  time,  the 
Defect  Measure  of  the  particles  can  be  computed  during  the  simulation  and  when 
the  Defect  Measure  levels  off,  the  simulation  can  be  cut  short.  Figure  5.5  shows  the 
Defect  Measure  plotted  against  the  time  step  for  a  single  simulation.  From  time 
step  110  until  the  end  of  the  simulation  at  time  step  150,  the  Defect  Measure  stays 
relatively  constant,  indicating  that  the  simulation  could  have  been  cut  off  at  time 
step  110.  Naturally,  the  Defect  Measure  would  not  need  to  be  computed  at  every 
time  step,  and  not  at  the  beginning  of  the  simulation.  Shortening  the  simulation 
time  in  this  way  in  an  optimization  scheme  would  speed  up  the  procedure. 

5.3  Quantifying  Robustness  of  Potentials 

A  good  potential  for  the  self-assembly  of  particles  will  be  robust  to  uncertainty  in 
the  parameters  of  the  potential,  the  density  of  particles  and  the  cooling  schedule. 
The  larger  the  range  of  values  over  which  the  final  lattice  formed  is  acceptably  good, 
the  more  robust  the  potential  is. 

There  are  two  ways  to  define  what  an  acceptably  good  lattice  is.  Lattices  that 
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(a)  (b) 

Figure  5.4:  The  two  components  of  the  Geometric  Defect  Measure,  (a)  Nearest 
Neighbour  Lines  Measure  versus  parameter  a2-  (b)  Angle  Measure  versus  parameter 
02. 


have  a  Defect  Measure  below  a  certain  value  can  be  accepted  as  good  lattices.  The 
actual  value  of  the  Defect  Measure  chosen  for  this  purpose  will  depend  on  the  weights 
assigned  to  the  different  types  of  defects.  This  is  because  the  value  of  the  Defect 
Measure  for  a  lattice  only  has  meaning  with  respect  to  another  value  determined 
with  the  same  weights. 

The  second  way  to  define  the  acceptably  good  lattices  is  by  using  Step  2  of  the 
procedure  to  compute  the  Defect  Measure.  If  the  lattice  can  be  identified  by  the 
algorithm  to  be  of  the  same  type  of  lattice  as  the  target  lattice,  then  it  is  an  ac¬ 
ceptably  good  lattice.  Such  lattices  will  have  a  majority  of  particles  that  have  the 
correct  number  of  nearest  neighbours,  next  nearest  neighbours  and  third  nearest 
neighbours.  This  definition  of  a  good  lattice  works  best  for  lattices  with  periodic 
boundary  conditions  or  lattices  with  many  more  inner  particles  than  boundary  par¬ 
ticles. 

Figure  5.6  plots  the  Defect  Measure  versus  density  of  particles  for  two  honey¬ 
comb  potentials:  Rcchtsman's  potential  and  a  piecewise  polynomial  potential.  The 
weights  of  Defect  Measure  1  were  used  in  the  computation.  Note  that  15  points  were 
omitted  from  Figure  5.6(a).  These  points  all  had  a  Defect  Measure  greater  than  495 
and  a  density  greater  than  1.  Eaeh  point  in  the  plot  corresponds  to  a  simulation 
of  65  particles  in  a  domain  without  periodic  boundary  conditions.  There  were  1000 
simulations  with  densities  given  by  a  Gaussian  distribution  with  mean  of  0.8  and 
standard  deviation  of  0.1. 

Defining  acceptably  good  lattices  to  be  those  having  a  Defect  Measure  less  than 
30,  Rechtsman’s  potential  yields  good  lattices  from  densities  of  0.6334  to  0.9578. 
The  polynomial  potential  yields  good  lattices  over  a  range  of  densities  from  0.6884 
to  1.0690.  This  corresponds  to  a  spread  of  0.3244  for  Rechtsman’s  potential  and  a 
spread  of  0.3806  for  the  polynomial  potential.  Thus  the  polynomial  potential  can 
be  said  to  be  more  robust  to  uncertainty  in  the  density  of  particles.  It  forms  good 
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Figure  5.5:  Defect  Measure  versus  time  for  a  single  simulation. 


lattices  over  a  wider  range  of  densities. 

The  sensitivity  of  potentials  to  the  parameters  in  the  potential  and  the  cooling 
schedule  can  be  examined  by  a  similar  procedure. 

5,4  Identification  of  Clusters  to  be  treated  as  Rigid  Bodies 

The  self-assembly  of  particles  with  pairwise  isotropic  in  ter- particle  potentials  into 
a  lattice  requires  long  and  expensive  simulations.  The  temperature  of  the  particles 
must  be  decreased  slowly  to  allow  a  lattice  to  form.  One  way  to  speed  up  such 
simulations  is  to  identify  particles  that  have  already  formed  a  lattice  structure  and 
treat  these  particles  as  a  rigid  body,  thus  decreasing  the  number  of  evaluations  of 
the  potential  function.  This  procedure  works  best  for  lattices  that  form  nuclei  that 
grow  and  join  up  to  form  a  lattice.  The  triangular  lattice  is  ideal  because  of  its 
stability  once  formed.  The  polynomial  potential  discussed  above  for  the  honeycomb 
potential  also  forms  in  this  way. 

A  cluster  of  particles  that  may  be  treated  as  a  rigid  body  can  be  identified  by 
observing  how  much  the  inter- particle  distances  change  over  time.  If  these  distances 
have  not  changed  significantly  for  an  appropriate  length  of  time  then  the  particles 
are  most  likely  in  the  lattice  configuration  and  may  be  treated  as  a  rigid  body. 

However,  there  may  be  some  defects  in  this  cluster  of  particles.  A  lattice  with  a 
defect  is  typically  less  stable  than  a  lattice  free  of  defects.  Over  time,  the  particles 
will  try  to  eliminate  the  defect  or  move  it  towards  the  edge  of  the  otherwise  well- 
formed  cluster.  This  takes  more  time  than  the  free  movement  of  particles  that  arc 
not  a  part  of  a  cluster.  So  a  cluster  that  has  a  defect  may  be  identified  as  a  cluster 
to  treat  as  a  rigid  body  because  the  inter-particle  distances  have  not  changed  much. 

To  allow  defects  within  clusters  of  particles  to  be  eliminated,  the  Defect  Measure 
can  be  calculated  for  each  candidate  rigid  body  particle.  Those  particles  with  a  high 
Defect  Measure  (those  close  to  the  defect)  would  then  not  be  included  in  the  cluster 
of  particles  to  be  treated  as  a  rigid  body. 
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(a)  Rechtsman’s  potential 


(b)  Polynomial  potential 


Figure  5.6:  Defect  Measure  versus  density  of  particles  for  (a)  Rechtsmaii’s  potential 
and  (b)  a  piecewise  polynomial  potential.  Each  point  corresponds  to  one  simulation 
of  65  particles  in  a  domain  without  periodic  boundary  conditions.  The  weights 
corresponding  to  Defect  Measure  1  were  used. 


This  application  of  the  Defect  Measure  has  not  yet  been  implemented.  See  Sun 
Hwan  Lee’s  work. 
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(CO)i_T(Ar),  mixtures  physisorbed  on  graphite  experimentally  display  a  novel  phenomenon  of 
increasing  phase  transition  temperature  (stabilizing  the  system)  with  increasing  Ar  impurity  con¬ 
centration  or  uncertainty  (H.  Wiechert  and  K.-D.  Kortmann,  Surf.  Sci.  441  (1999)].  We  develop  a 
2D  Ising  model  that  accurately  captures  the  phase  transition  and  its  temperature  dependence.  The 
anomaly  in  transition  temperature  is  due  to  formation  of  pinwheel  regions  of  CO  around  Ar  atoms. 
The  dilemma  of  whether  the  ground  state  is  head-to-head  or  head-to-tail  ordered  is  reconciled  in 
favor  of  the  latter.  The  phase  transition  curve  in  the  presence  of  uncertainty  in  Ar  impurity  is 
computed  using  Monte  Carlo  (MC)  and  Probabilistic  Collocation  Method  (PCM).  PCM  computes 
the  first  two  moments  %  2000  times  faster  than  MC. 

PACS  numbers:  68.35.Rh,  68.55.Ln,  OS.SO.-f-q,  02.70.Jn 


Uncertainty  is  an  important  factor  in  the  design  of  physical  models.  Usually  significant  effort  is 
needed  to  minimize  uncertainty  in  the  output  of  a  model  subject  to  input  uncertainty.  Phase  transi¬ 
tions  in  statistical  thermodynamics  of  condensed  matter  are  some  of  the  most  vivid  manifestations 
of  the  effect  of  uncertainty  on  the  state  of  a  system.*’^  In  particular,  defects  in  the  form  of  vacancies, 
interstitial  and  quenched  impurities  give  rise  to  random  fields  which  are  known  to  catalyze  phase 
transitions.^*'*  Disorder  and  nonequilibrium  effects  are  known  to  modify  structural  phase  transitions 
in  pure  periodic  systems.^  Even  small  amounts  of  impurities  in  monolayers  of  adsorbed  gases  can 
induce  significant  changes  in  the  phase  behavior  and  phase  transitions.  Impurities  tend  to  distort 
the  sublattices  of  the  adsorbed  phase,  and  hence  lead  to  a  phase  transition  into  an  “intermediate” 
phase  significantly  prior  to  that  observed  for  a  pure  monolayer.®  Low  temperature  phase  transitions 
in  some  systems  can  be  regarded  as  realizations  of  two-dimensional  (2D)  Ising  systems.^ 

Recently,  Carbon  Monoxide  (CO)  monolayers  with  Argon  (Ar)  impurities  physisorbed  on  graphite 
have  been  studied  experimentally  and  found  to  exhibit  unique  physical  properties.'**^  When  adding 
Ar  impurities  to  head-tail  ordered  CO  monolayers,  the  order  in  the  system  is  slowly  destroyed  and 
the  phase  transition  is  found  to  be  completely  suppressed  when  the  impurity  concentration  reaches 
J5S  7%.'*  Unlike  any  other  known  physical  system,  the  disorder  induced  in  a  CO  monolayer  by  Ar 
impurities  results  in  a  higher  phase  transition  temperature,  thereby  stabilizing  the  head-tail  ordered 
phase.'*  The  phase  transition  of  interest  (called  head-tail  ordering  transition)  occurs  at  ^  5.18K.®’*® 
In  this  Rapid  Communication  we  develop  a  model  of  the  CO-Ar  system  and  explain  the  origin  of 
the  observed  phenomena. 

CO-Ar  mixtures  physisorbed  on  graphite  can  be  considered  as  experimental  realizations  of  2D  Ising 
models.  In  the  following  we  design  an  Ising  model  that  captures  the  head- tail  ordering  transition 
along  with  the  anomalous  shift  in  the  transition  temperature  with  increasing  Ar  concentration. 
The  stabilization  of  the  phase  with  uncertainty  is  successfully  captured  by  the  Ising  model  when 
the  experimentally  observed  pinwheel  structure'*  of  CO  molecules  around  the  Ar  sites  is  correctly 
modeled  (see  Fig.  1).  We  study  the  phase  transition  curve  as  a  function  of  Ar  concentration  in 
the  presence  of  uncertainty.  The  unpredictability  in  the  exact  concentration  of  Ar  atoms  is  another 
source  of  uncertainty.  The  latter  uncertainty  transforms  the  phase  transition  curve  into  a  phase 
transition  region,  which  can  be  captured  by  computing  the  moments  of  phase  transition  region 
at  every  nominal  concentration  of  Ar.  The  moments  are  obtained  using  Monte  Ceirlo  (MC)  and 
Polynomial  Chaos  (PCH)  techniques.  The  two  methods  are  compared,  and  it  is  found  that  PCI  I 
captures  the  moments  of  the  uncertain  phase  transition  curve  »  2000  times  faster  than  standard 
MC. 
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(a) 


FIG.  1;  CO-Ar  on  graphite,  (a)  Pinwheel  structure  formed  around  the  Ar  impurities,  (b)  Flip  of  CO 
molecule  in  Metropolis- Hastings. 


We  model  the  potential  energy  of  the  system  as  a  sum  of  pairwise  interactions  between  CO 
molecules  and  their  six  nearest  neighbors:  U  =  where  N  is  the  total  number  of  CO 

molecules.  Pairwise  interactions  f/,j  take  into  account  electrostatic  interactions  of  point  charges  on 
both  molecules,  Van  der  W^aals  interactions  between  the  two  molecules  and  interactions  with  the 
graphite  substrate.  The  center  of  mass  of  each  molecule  is  shifted  by  a  distance  dc  from  the  center 

of  the  hexagonal  lattice  cell  in  which  it  is  located  (see  Fig.  1). 

To  simulate  the  system  at  different  temperatures  we  employed  the  Metropolis- Hastings  algorithm. 
A  CO  site  is  picked  at  random  and  its  spin  is  flipped  (i.e.  C  and  O  are  swapped,  see  Fig.  1). 
If  the  total  energy  of  the  modified  system  decreases,  the  spin  flip  is  accepted,  otherwise  the  flip  is 
accepted  with  probability  associated  with  the  Boltzmann  distribution.^*  For  all  simulations  reported 
in  this  Rapid  Communication,  2  x  10"*  flips  are  performed  per  CO  site.  At  each  temperature,  the 
fluctuation  in  energy  U  gives  us  the  heat  capacity  (C,,).**  The  center  of  mass  offset  dr  is  a  very 
important  parameter  of  the  model.  In  the  absence  of  impurities,  we  observe  head-to-tail  ordering 
for  dr  =  0.202,  while  dc  =  0.205  results  in  head-to-head  ordering.  Both  values  of  dc  correspond  to 
the  experimentally  observed  position  of  the  heat  capacity  peak. 

All  results  are  obtained  for  a  1  O'* -site  Ising  system.  The  impurities  are  chosen  randomly  on 

the  two-dimensional  lattice  with  the  restriction  that  no  CO  molecules  can  have  more  than  one 

impurity  in  their  six  nearest  neighbors.  The  locations  of  impurities  on  the  lattice  are  found  to 
make  no  difference  for  the  transition  temperatures.  When  the  center  of  mass  offset  dr  lies  outside 
of  the  interval  [0.195;  0.212],  the  system  exhibits  “classical”  behavior  with  high  phase  transition 
temperatures  which  decrease  with  increased  impurity  concentration  as  the  system  becomes  less 
stable.  For  the  head-to-head  ordered  monolayer  with  dr  =  0.205,  the  phase  transition  temperature 
does  not  depend  on  the  impurity  concentration.  Finally,  for  the  head-to-tail  ordered  monolayer 
with  dr  =  0.202,  the  phase  transition  temperature  shifts  to  higher  temperatures  with  the  inclusion 
of  impurities.  In  addition  to  the  fact  that  our  model  is  able  to  explain  the  anomalous  increase  in 
transition  temperature  with  impurity  concentration,  it  also  reconciles  the  dilemma  of  whether  the 
ground  state  is  head-to-head  or  head-to-tail  ordered  in  favor  of  the  latter."* 

For  each  concentration  of  Ar,  a  heat  capacity  curve  is  computed  with  dc  =  0.202  (see  Fig.  2).  It  can 
be  seen  in  Fig.  2  that  the  effect  of  Ar  impurities  on  the  Ct,  curves  match  experimental  observations."* 
Both  the  suppression  of  the  heat  capacity  curve  along  with  its  anomalous  shift  to  higher  transition 
temperatures"*  are  captured.  To  the  best  of  our  knowledge,  the  Ising  model  implemented  in  this  paper 
is  the  first  model  that  captures  the  anomalous  shift  of  the  phase  transition  temperature  caused  by 
formation  of  pinwheel  structures  around  Ar  impurity  sites. 

From  Fig.  2  the  transition  temperatures  can  be  plotted  against  the  concentration  of  Ar  (we  pick 
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10000  CO  molecules 


FIG.  2:  Heat  capacity  C„  for  different  concentrations  of  Ar  impurities. 


—  Ising  Model  Phase  Transition 
— Quadratic  Fit  to  Phase  Transition 

—  Experimental  Phase  Transition 

—  Error  Bar 


0.005  0.01  0.015  0.02  0.025  0.03 

Argon  Density 


FIG.  3:  Phase  transition  temperature  as  a  function  of  Ar  impurity  density  extracted  from  the  Ising  model 
(red  curves)  and  its  quadratic  fit  (blue  curve).  Experimental  data  (black  curve)  and  error  bars  are  taken 
from  Ref.  4. 


the  transition  temperature  at  the  peak  for  every  curve  in  Fig.  2).  The  transition  temperatures  with 
increasing  A  r  density  2Lre  plotted  in  Fig.  3,  demonstrating  the  anomalous  shift  found  experi  men  tally. 
The  noisy  curve  obtained  from  the  Ising  model  (see  Fig.  3)  is  approximated  by  a  quadratic  fit  to  ease 
the  computation  of  the  phase  transition  in  the  presence  of  uncertainty.  Moreover,  our  theoretical 
predictions  are  found  to  lie  within  experimental  error  of  Ref.  4  as  shown  in  Fig.  3. 

In  the  experiments,  there  is  always  uncertainty  in  the  concentration  of  impurities.  This,  in  turn, 
makes  the  phase  transition  curve  uncertain.  Hence,  different  experimental  realizations  of  CO-Ar 
mixtures  on  graphite  will  give  rise  to  slightly  different  phase  transition  curves.  In  the  following  we 
quantify  the  variability  in  the  phase  transition  curve  in  the  presence  of  uncertainty  in  the  concen¬ 
tration  of  Ar  impurities.  To  compute  the  variability  we  use  two  different  approaches.  The  first 
approach,  the  MC  technique,  is  a  fairly  standard  statistical  analysis  method.  This  method  is,  how¬ 
ever,  very  inefficient.  A  significant  speedup  in  the  computation  of  the  uncertainty  in  the  phase 
transition  curves  can  be  obtained  by  employing  PCH  techniques. 

For  each  value  A^o  of  Ar  concentration  in  Fig.  3  we  assume  that  the  concentration  of  Ar  is 
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distributed  binomially^'*  with  mean  at  A'o  and  standard  deviation  of  3%  (e  =  0.03).  The  parameters 
of  the  binomial  distribution  N  and  p  can  be  found  from  the  following  system: 

Np  =  h\) 

Np{l-p)  =  (cKof.  (1) 

For  each  value  of  mean  Ar  concentration,  we  sample  the  corresponding  binomial  distribution  in  Ar 
concentration  using  MC  techniques  and  compute  the  mean  and  standard  deviation  of  the  transition 
temperature.  The  results  of  the  MC  procedure  are  shown  in  Fig.  4  for  10^  samples  for  each  binomial 
distribution.  As  expected,  the  mean  transition  curve  is  the  same  as  the  quadratic  fit  in  Fig.  3.  The 
region  between  the  outer  curves  in  Fig.  4  is  one  standard  deviation  around  the  mean  transition 
temperature.  As  the  concentration  of  Ar  atoms  increases,  the  uncertainty  in  the  phase  transition 
temperature  also  increases.  Although  MC  methods  succeed  in  computing  the  phase  transition  curve 
in  the  presence  of  uncertainty,  a  very  large  number  of  samples  (10^)  are  necessary  for  a  reliable 
estimate. 

PCH  techniques  are  used  to  quantify  output  uncertainty  by  expanding  the  output  random  variable 
of  interest  in  an  optimally-chosen  orthogonal  basis. Let  us  consider  the  following  system: 

X  =  /(x,A),  (2) 

where  x  is  the  system  output  and  A  is  a  vector  of  uncertain  system  parameters  with  associated 
probability  distribution  i/;(A).  In  PCH  the  output  random  variable  is  expressed  as 

x{t;  A)  =  ao(t)\l;o{X)  +  ai{t)xpi  (A)  +  . . .  (3) 

Here  {V^i(A)  :  i  G  1, . . . ,  oo}  forms  an  orthogonal  basis  with  respect  to  ic(A). 

The  coefficients  ai{t)  can  be  determined  using  Galerkin  projections*^  when  the  equations  of  the 
system  are  explicitly  known.  In  the  case  of  the  Ising  model,  the  phase  transition  temperature  is  a 
random  variable  and  /  is  the  Metropolis-Hastings  code.  Since  /  is  not  known  explicitly,  one  can 
apply  Probabilistic  Collocation  Methods  (PCM),*®  where  the  output  random  variable  is  expanded 
using  Eq.  3.  However,  the  system  parameters  are  sampled  using  zeros  of  a  polynomial  orthogonal 
to  the  basis  used  in  expansion  Eq.  3  (typically  if  the  order  of  expansion  is  n,  is  chosen).  A 

Lagrange  interpolating  polynomial  is  passed  through  the  output  and  the  resulting  moments  of  the 
distribution  are  computed.  The  orthogonal  polynomials  corresponding  to  the  binomial  distribution 
are  the  Krawtehouk  polynomials  The  zeros  of  the  polynomials  Kji  take  non-integer  values. 

Since  the  number  of  Ar  atoms  has  to  be  discrete,  we  use  the  quadratic  fit  in  Fig.  3  to  compute  the 
phase  transition  temperature  at  a  fractional  number  of  Ar  atoms. 

The  results  of  using  PCM  can  be  seen  in  Fig.  4.  PCM  captures  the  mean  transition  temperature 
curve  along  with  the  one  standard  deviation  curves  exceedingly  well  with  just  4  input  samples.  The 
error  in  the  mean  and  variance  at  the  Ar  concentration  of  3%  can  be  seen  in  Figs.  5(a)  and  5(b). 
The  results  from  PCM  are  obtained  with  the  same  magnitude  of  error  as  MC  2000  times  faster. 

In  this  Rapid  Communication  we  studied  the  effect  of  argon  impurities  on  the  head-tail  ordering 
phase  transition  in  CO  monolayers  physisorbed  on  graphite.  We  developed  an  Ising  model  that 
captures  the  head- tail  ordering  transition  in  CO-Ar  mixtures  in  agreement  with  experimental  data. 
The  unique  physical  properties  of  the  CO-Ar  system  have  been  explained  and  attributed  to  the 
formation  of  pinwheel  regions  of  CO  around  the  Ar  impurities.  To  quantify  the  uncertainty  in  the 
number  of  Ar  atoms,  we  have  applied  PCM  and  found  it  to  be  far  superior  to  MC  in  this  problem. 
This  approach  can  be  used  to  quickly  bound  the  variation  in  phase  transition  curves  when  the 
impurity  concentration  is  not  known  accurately. 
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FIG.  4:  Comparii>on  of  the  mean  and  one  standard  deviation  phase  transition  curves  computed  using  Monte 
Carlo  and  Polynomial  Chaos. 


FIG.  5:  Comparison  of  the  number  of  samples  needed  by  Monte  Carlo  (blue  dots)  and  Polynomial  Chaos 
(red  dots)  to  obtain  mean  (left)  and  variance  (right)  of  the  phase  transition  temperature  in  the  CO-Ar 
system  with  3%  of  Ar  impurities. 
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Abstract 

The  goal  of  this  paper  is  to  develop  modeling  techniques  for  complex  systems  for  the 

purposes  of  control,  estimation,  and  inference: 

(i)  A  new  class  of  Hidden  Markov  Models  is  introduced,  called  the  finite-rank  optimal- 
prediction  (FRO)  model.  It  is  similar  to  the  Gaussian  mixture  model  in  which 
the  actual  marginal  distribution  is  used  in  place  of  a  Gaussian  distribution.  This 
structure  le2ids  to  simple  learning  algorithms  to  find  an  optimal  model. 

(ii)  The  FRO  model  provides  a  unification  of  other  modeling  approaches  including  the 
projective  methods  of  Shannon,  Mori  and  Zwanzig,  and  Chorin,  as  well  as  a  version 
of  the  binning  technique  for  Markov  model  reduction. 

(iii)  Several  general  applications  are  surveyed,  including  optimal  control,  and  the  dy¬ 
namical  analysis  of  complex  systems  via  Markov  spectral  theory.  Computation 
of  the  spectrum,  or  solutions  to  dynamic  programming  equations  are  possible 
through  a  finite  dimensional  matrix  calculation  without  knowledge  of  the  underly¬ 
ing  marginal  distribution  on  which  the  model  is  based. 
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1  Introduction 

Complex  systems  can  be  found  throughout  engineering,  and  the  social,  physical  or  life  sciences, 
and  every  academic  community  that  must  confront  complexity  has  developed  specialized  tools 
for  modeling  complex  systems.  In  this  paper  we  survey  some  of  these  methods,  and  provide  a 
unifying  framework  for  the  purposes  of  estimation  and  control. 

Markov  models  are  adopted  as  a  basic  model  since  they  can  capture  a  range  of  non- stationary 
behavior.  They  are  a  valuable  modeling  technique  even  for  models  that  are  not  Markovian,  This 
observation  is  initially  due  to  Claude  Shannon. 

Shannon  introduced  the  idea  of  low  dimensional  Markov  models  to  replicate  features  of 
English  language.  This  appears  as  the  motivation  for  the  notion  of  entropy  in  his  famous 
1948  paper  A  mathematical  theory  of  communication,  which  is  regarded  as  the  birth  of  modern 
information  theory  [26].  These  early  ideas  led  to  modern  techniques  in  source  coding  (e.g.,  the 
Lempel-Ziv  algorithm  used  in  every  computer  for  compression  [18]). 
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The  main  idea  is  simply  described  as  follows.  Let  Z  denote  a  discrete- time  stochastic 
process  for  which  we  seek  a  Markovian  description,  evolving  on  a  subset  Z  of  Euclidean  space. 
For  the  purposes  of  modeling  it  is  assumed  that  the  process  is  stationary.  Shannon’s  (first-order) 
Markov  model  is  obtained  using  Baye’s  rule  to  construct  a  one-step  transition  kernel, 


T(z.A) 


P{Z(t)edz,  Z{t  +  i)eA} 
P{Z{t)edz} 


zez,  Ac  B{Z). 


(1) 


where  the  ratio  denotes  the  Radon-Nikodym  derivative.  That  is,  T  describes  the  average  dy¬ 
namics  of  Z  in  steady-state.  The  Markov  model  with  transition  kernel  T  also  captures  a  large 
part  of  the  steady-state  behavior:  It  follows  from  the  definitions  that  the  marginal  distribution 
of  Z  is  invariant  for  T :  Letting  /z  denote  the  marginal,  we  have,  for  any  set  A  C  ^(Z), 


^l{A)  =  J  ^{dz)T{z,A)  (2) 

Or,  in  operator-theoretic  notation,  f.iT  = 

Shannon’s  model  was  developed  independently  for  continuous-time  processes  in  the  physics 
research  community:  It  is  a  component  of  the  famous  Mori-Zwanzig  projection  method  [22,  27], 
The  model  (1)  also  coincides  with  the  Markov  model  developed  by  Chorin  (see  e.g.  [8,  7]).  In 
this  w'ork  the  Markov  model  is  said  to  exhibit  optimal  prediction  due  to  the  solidarity  of  the 
steady-state  marginals  captured  in  the  invariance  equation  (2). 

While  providing  powerful  and  general  approaches  to  model  reduction.  Shannon’s  approach 
can  still  result  in  a  highly  complex  model.  In  particular,  if  Z  is  not  finite,  then  the  Markov 
model  with  transition  kernel  T  can  be  regarded  as  an  infinite-dimensional  nonlinear  dynamical 
system.  To  avoid  this  complexity  we  seek  a  finite  dimensional  setting.  The  approach  taken 
in  this  paper  is  to  enforce  the  optimal  prediction  property  on  a  finite-dimensional  space  of 
functions.  There  are  many  ways  to  enforce  this  constraint  —  Motivation  for  the  class  of  models 
adopted  in  this  paper  comes  from  recent  spectral  theory  of  Markov  processes. 

Various  theories  from  Markov  processes  suggest  a  model  whose  transition  kernel  has  finite 
rank:  For  a  collection  of  functions  {/li}  and  measures  {/i,}  the  kernel  can  be  expressed  as  the 
finite  sum, 

m 

T  hi  (3) 

i=l 

SO  that  T(z,A)  =  ^hi{z)fj,i{A)  for  any  z  and  A.  For  example,  if  Z  is  a  finite  state  space 
Markov  model  whose  transition  matrix  has  distinct  eigenvalues,  then  (fi)  holds  with  {hi}  and 
{/ij}  right  and  left  eigenvectors.  For  general,  infinite  dimensional  models  it  is  still  possible  to 
obtain  a  reduced  order  model  based  on  the  eigenvectors  corresponding  to  eigenvalues  near  unity. 
Based  on  the  sign  structure  of  the  eigenvectors,  it  is  possible  to  approximate  a  complex  Markov 
model  by  a  much  simpler  hidden  Markov  model  (HMM)  [1  Ij.  This  construction  is  rooted  in  the 
asymptotic  theory  of  Freidlin  and  Wentzell  [12,  (>],  the  theory  of  quasi-stationarity  for  Markov 
chains  [11],  and  the  approximate  modeling  approaches  based  on  spectral  theory  developed  in 
[13,  9]  —  see  also  recent  approaches  in  [15,  19]. 

Finite  rank  Markov  models  with  transition  kernel  of  the  form  (3)  are  used  to  approximate 
complex  Markov  processes  in  [3,  17]  for  the  purposes  of  verifying  the  existence  of  a  spectral 
gap,  and  for  establishing  limit  theory  such  as  large  deviations. 

Other  common  approaches  to  model  reduction  also  lead  to  a  finite  rank  Markov  model  of 
this  form.  If  Y  is  the  output  process  of  an  HMM  with  finite  state  process  /,  then  the  joint 
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process  (I.Y)  is  Markov  with  a  finite-rank  transition  kernel,  even  if  the  observations  evolve  in 
a  complex  space. 

The  use  of  binning  to  create  a  finite-dimensional  Markov  model,  as  in  [Kl,  9],  results  in  a 
finite  state-space  model,  or  a  simple  refinement  (given  in  (8))  leads  to  a  finite-rank  model  of 
the  form  (3). 

The  approach  advocated  in  this  paper  combines  all  of  the  points  of  view  surveyed  above, 
based  on  the  finite-rank  optimal-prediction  (FRO)  model,  introduced  here  for  the  first  time. 
Let  {ri,Si  :  I  <  i  <  m}  denote  measurable  functions  on  Z,  each  in  and  for  a  collection 

of  constants  {B|j}  denote, 

m 

sh(2o,2i)  =  (4) 

ij=l 

The  FRO  model  is  Markovian,  whose  transition  kernel  is  finite  rank  with  density  Se, 

T^{zo,A):=  s»(zo,zi)iJ.{dzi).  (5) 

JzieA 

The  subscript  in  the  density  is  used  to  stress  that  this  is  to  be  learned.  The  task  of  learning  is 
greatly  simplified  by  choosing  to  avoid  estimating  the  entire  dynamics. 

The  FRO  model  has  unique  features,  not  found  in  generic  HMM  models,  that  are  developed 
in  this  paper.  To  summarize, 

(i)  The  structure  (3.2)  in  which  /i  is  the  (unknown)  marginal  distribution  of  the  vector  Z(t) 
leads  to  a  simple  characterization  of  the  best  Sh  through  L2  or  i  n  for  mation-th  core  tic 
methods.  Complexity  of  computation  of  the  best  parameter  is  greatly  reduced  when 
compared  to,  say,  the  EM  method  [18]. 

(ii)  In  particular,  this  class  of  models  can  be  chosen  to  capture  the  optimal  prediction 
property:  For  a  given  collection  of  functions  {0i}  in  L2(fi)  we  can  choose  0  =  0*  to 
guarantee, 

+  1))1  =  ElMZimjiZit  +  1))],  ij  =  1, . . .  ,m,  (6) 

where  the  left  hand  side  denotes  the  expectation  with  respect  to  the  model  obtained 
using  0*,  and  the  right  hand  side  is  the  actual  expectation  in  steady  state.  This  is  a 
generalization  of  the  optimal  prediction  property  described  by  Chorin  for  the  Markov 
models  of  Mori  and  Zwanzig  [8,  7,  22,  27].  In  fact,  these  models  were  introduced  by 
Shannon  for  the  first  time  in  [26]  —  Some  history  is  contained  in  Section  2. 

(iii)  The  FRO  model  can  be  adapted  to  obtain  a  model  based  on  binning,  as  in  [13,  9],  while 
again  preserving  the  optimal  prediction  property  (6). 

(iv)  Knowledge  of  the  full  marginal  p,  is  unnecessary  in  all  applications  of  interest  in  this 
paper.  In  particular, 

(a)  Only  finite-dimensional  statistics  (means  and  covariances)  are  required  to  ob¬ 
tain  the  optimal  parameter  in  (3.2).  These  statistics  are  easily  estimated 
through  Monte-Carlo  techniques. 
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(b)  It  is  shown  in  Proposition  3.3  that  computation  of  the  spectra  of  the  kernel  T  is 
also  possible  through  knowledge  of  finite-dimensional  statistics.  An  application 
of  this  result  is  used  in  Section  5,  which  contains  a  detailed  treatment  of  the 
prediction  of  phase  transitions  in  molecular  models. 

(c)  Similarly,  the  solution  to  dynamic  programming  equations  that  arise  in  op¬ 
timization  as  well  as  simulation  can  be  computed  based  on  easily  estimated 
finite-dimensional  statistics. 

We  believe  that  the  FRO  model  is  ideal  for  applications  to  optimization,  distributed  control, 
simulation  variance  reduction,  as  well  as  prediction  of  phase  transitions. 

The  remainder  of  the  paper  is  organized  as  follows.  Contained  in  the  next  section  is  a  review 
of  Shannon’s  approach  to  Markov  modeling,  and  results  demonstrating  how  more  recent  meth¬ 
ods  fall  into  Shannon’s  framework.  Section  3  contains  a  development  of  the  finite-rank  Markov 
model.  Several  different  techniques  are  introduced  to  obtain  a  model  exhibiting  the  optimal 
prediction  property  on  a  finite-dimensional  function  class.  Section  4  contains  aa  Monte-Carlo 
methods  to  compute  the  parameters  in  a  finite- rank  Markov  model  and  provide  a  statistical 
analysis  of  the  estimates. 

Section  3  contains  an  in-depth  application  of  the  spectral- theoretic  techniques  to  the  phase- 
transition  prediction  problem  of  molecular  dynamics. 

Conclusions  and  extensions  are  summarized  in  Section  7. 

Notational  conventions  An  upper  case  symbol  such  as  X  denotes  a  random  variable,  and 
lower  case  x  deterministic.  Bold  italic  X  indicates  a  stochastic  process. 

To  indicate  relative  dimension  we  let  #  denote  a  very  large  integer.  Hence,  if  we  say  that 
‘‘x  evolves  on  R^”  this  signifies  that  x  is  a  deterministic  process  evolving  on  a  state  space  so 
large  that  simulation  is  extremely  difficult  on  an  ordinary  computer. 

We  let  z  oT  Z  denote  a  process  on  a  simpler  state  space.  A  Markov  model  is  constructed 
to  approximate  the  behavior  of  Z.  Its  state  process  is  denoted  Z,  each  evolving  on  the  state 
space  Z. 

2  Shannon’s  Markovian  projection 

In  the  treatment  of  deterministic  dynamical  systems,  model  reduction  is  typically  addressed 
through  singular- value  decomposition  to  eliminate  ‘fast  variables’.  Hence  the  choice  of  coarse 
variables  in  a  reduced-complexity  description  is  specified  by  relative  dynamics.  In  the  probar 
bilistic  framework  considered  here  we  have  much  greater  freedom:  Regardless  of  what  variables 
are  chosen  to  build  a  Markov  model,  the  resulting  nonlinear  system  is  stable  (see  Proposition  2. 1 
that  follows).  Moreover,  the  model  replicates  exactly  whatever  statistics  are  input  as  constraints 
in  the  construction  of  the  transition  kernel. 

The  following  result  is  based  on  Shannon’s  construction  (1).  It  is  a  component  of  the  model 
reduction  techniques  pioneered  by  Mori  and  Zwanzig  in  the  area  of  statistical  mechanics  [22,  27], 
The  conclusion  of  the  proposition  is  described  by  Chorin  as  optimal  prediction  since  the  model 
T  captures  exact  marginal  statistics  of  Z. 

Proposition  2.1.  Suppose  that  the  Radon-Nikodym  derivative  (1)  exists  for  each  z  and  A  to 
define  a  transition  kernel  on  Z  x  B(Z).  That  is,  T(  • ,  A)  is  a  measurable  function  on  B{Z)  for 
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each  A  €  I3{Z),  and  T{z^  ♦ )  is  a  probability  measure  on  B{2.)  for  each  z  gZ.  Then  the  Markov 
chain  with  this  transition  kernel  describes  these  aspects  of  the  stationary  process  Z: 

(i)  One-step  dynamics:  T{z,  A)  =  P{Z{t  -h  1)  €  A  |  Z{t)  =  z},  z  e  A  G  13{Z). 

(ii)  Steady-state:  The  probability  p  is  invariant  for  T, 

p{A)=  [  p{z)T{z.A),  Ag3(Z). 


This  result  is  a  trivial  consequence  of  the  definitions,  yet  its  implications  are  surprisingly 
rich.  A  roadblock  to  its  application  is  that  the  transition  kernel  T  is  not  known.  Moreover,  in 
general  it  remains  an  infinite  dimensional  object,  in  which  case  learning  the  entire  transition 
kernel  is  not  feasible.  In  Section  3  we  turn  to  kernels  of  finite  rank  to  approximate  the  optimal 
prediction  model.  First  we  consider  some  special  cases  in  which  the  form  of  the  approximate 
model  is  relatively  transparent. 


2.1  Model  construction  using  binning 

Let  {Xi,...,Xn}  denote  a  partition  of  the  state  space  for  the  complex,  stationary  process  X: 
These  sets  are  assumed  disjoint,  with  IJXi  =  R^.  This  partition  is  used  to  define  a  coarse 
variable  that  indicates  the  particular  partition  that  X(t)  occupies  at  each  time  t. 

Let  Z  denote  the  integer- valued  process  taking  values  in  the  set  {1,2,  ....,n}  defined  by 
{Z(t)  =  i  iff  €  Xi,  1  <i<  m}.  Shannon’s  construction  has  a  simple  interpretation  in  this 
case  because  Z  takes  on  only  m  possible  values. 

Proposition  2.2.  With  Z  defined  using  a  state  space  partition,  the  transition  matrix  (1)  can 
be  expressed  for  zq  =  i  and  z\  =  j  by  the  ratio  of  the  probabilities, 


T{zo,zi) 


P{X(Q)  e  X,,  ;r(i)  e  x,} 
P{X{0)  e  X.} 


(7) 


That  is,  T{zo,zi)  =  (7r{Xi})  ’ 7r{dx)P{x,Xj),  where  tt  is  the  marginal  distribution  for 
X.  ‘  □ 


If  desired,  a  finite-rank  Markov  model  on  the  original  state  space  can  be  obtained  using  the 
transition  kernel. 


r{xo,dx,):=T{zo,zi)'^^^,  xoCX.,  xieX^.  (8) 

The  definition  (7)  is  used  in  [13,  9]  to  construct  a  finite  state  space  Markov  chain.  In  these 
papers  the  distribution  of  X(0)  is  arbitrary,  so  that  the  invariant  measure  for  T  will  not  be 
consistent  with  X. 
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2.2  Markov  spectral  theory 

The  binning  approach  can  be  justified  based  on  the  results  from  spectral  theory  surveyed  above: 
If  the  sets  {Xj}  are  selected  so  that  the  transition  times  are  approximately  geometrically  dis¬ 
tributed,  then  an  accurate  Markov  chain  approximation  can  be  obtained.  Results  in  [11,  21] 
show  that  the  geometric  distribution  approximation  holds  for  a  sampled  diffusion,  provided  the 
sets  (Xt)  are  sublevel  sets  of  the  associated  eigenfunctions. 

We  next  consider  a  more  general  setting:  Suppose  that  Z  is  an  m-dimensional  vector- valued 
function  of  a  Markov  chain  X  evolving  on  There  is  a  function  tp:  —>  R'”  such  that 

Z{t)  =  'ip{X{t))  for  each  t.  If  the  spectra  of  X  and  Z  coincide,  then  a  decomposition  of  the 
state  space  Z  =  R^  appears  justifiable  based  on  the  spectrum  of  T.  Under  special  conditions 
Shannon’s  model  does  capture  a  part  of  the  overall  spectrum. 

Proposition  2.3.  Suppose  that  h  is  an  eigenfunction  for  X  with  real  or  complex  eigenvalue  A, 
and  suppose  moreover  that  it  can  be  expressed  as  a  function  of'ip:  For  some  function  g:  R"^  —y  C, 

h{x)  -  i  e  R*. 

Then  g  is  also  an  eigenfunction  for  T  with  eigenvalue  A. 

Proof.  For  any  function  /:  R’^  ^  C  we  have  from  the  definitions  and  the  Markov  property, 

E[f{Z{t))g{Z{t  +  1))]  =  E[f{Z{t))h{X{t  +  1))]  =  E[f{Z{t))Ph(X(t))]. 

Based  on  the  eigenvector  equation  we  obtain,  E[f{Z{t))g{Z{t  +  1))]  =  E[f{Z{t)){Xg{Z{t)))]. 
The  optimal  prediction  property  implies  that  the  same  identity  holds  with  Z  replaced  by  Z, 

E[f(Z{t))g{Z(t  +  1))]  =  E[f(Z{t)){Xg{Z{t)))]. 

Prom  the  definition  of  conditional  expectation  we  conclude  that  E\g(Z{t-\-\))  \  Z(<)]  =  Xg{Z{t))y 
or  Tg  =  Xg.  □ 


3  Finite-rank  optimal-prediction  models 

In  this  section  we  introduce  several  approaches  to  model  construction  for  the  FRO  model  with 
transition  kernel  (3.2).  Stationarity  of  Z  is  assumed  throughout,  even  though  the  ultimate  goal 
is  to  construct  a  Markovian  approximate  model  whose  realizations  are  not  necessarily  stationary. 
Some  general  properties  of  finite  rank  models  are  summarized  in  Section  3.1. 

The  main  result  of  Section  3.2  shows  how  to  construct  0*  satisfying  the  optimal  prediction 
property  (()).  The  remainder  of  this  section  develops  optimization  techniques  for  the  construc¬ 
tion  of  a  model.  In  Section  3.3  a  parameter  is  found  that  minimizes  an  L2  error  criterion. 
It  is  shown  in  Proposition  3.4  that  an  optimizer  satisfies  the  optional  prediction  property  ((i) 
with  {4>i}  =  {r,},  provided  the  functions  {r^}  and  {mj}  coincide.  An  alternative  maximum 
likelihood  criterion  is  described  in  Section  3.4,  and  similar  conclusions  are  obtained. 

The  following  conventions  are  adopted  throughout  this  section.  For  any  R"^- valued  functions 
f,g  whose  components  lie  in  L2{pl)  we  denote  the  auto-correlation  functions, 

R{f{k)  =  E[fi(Z(0))gj{Z(k))],  =  keZ  (9) 
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or  in  matrix  form,  RL^{k)  =  E[f{Z{0))g’^{Z{k))].  When  /  =  p  we  write  Rl {k)  instead  of 
RM(k). 

Recall  that  //  denotes  the  marginal  distribution  of  Z.  We  let  denote  the  stationary  bivari¬ 
ate  distribution  describing  the  joint  statistics  of  {Z{t),  Z(<  -h  1)),  the  stationary  distribution 
for  and  the  stationary  bivariate  distribution  under  the  transition  law  That  is, 

fil(dzo,dzi)  :=  fi»{dzo)T^{zo,dzi)  (10) 

The  bivariate  distribution  possesses  a  density  with  respect  to  the  product  distribution  /x  x  /i, 
denoted  which  can  be  expressed  as  a  finite  sum  of  the  form, 

m 

pI{zo.z\)  =  (11) 

«j=i 

To  obtain  a  tractable  framework  for  approximation,  we  assume  throughout  the  remainder 
of  the  paper  that  the  functions  {ri.mj}  are  given.  The  functions  {sj}  in  (4)  are  obtained  from 
the  functions  {ri,mi}  via, 

m 

We  begin  with  some  comments  on  the  general  finite  rank  model. 

3.1  Properties  of  finite-rank  Markov  models 
Consider  a  general  transition  kernel  of  the  form, 

m 

T(z,  A)^Yi  «t(^)w(>l).  z&Z,  >16  B(Z),  (13) 

t=l 

where  {st}  are  non-negative  valued  functions,  and  {pi}  are  probability  measures.  Let  Z  denote 
the  Markov  chain  with  this  transition  kernel, 

P{Z(t  4-  1)  6  d  I  Z{t)  =  z}=  T(2,  A). 

Some  properties  of  T  and  Z  are  summarized  in  the  following: 

Proposition  3.1.  The  Markov  chain  Z  with  jinite-rank  transition  kemel  (13)  has  the  following 
propeHies: 

(i)  Z  is,  of  course,  a  Markov  chain  on  the  state  space  Z. 

(ii)  The  m-dimensional  stochastic  process  W (t)  =  (si(Z(t)), . . . ,  Sm(Z{t)))^ ,  t  >0,  is  a 
Markov  chain  on  . 

(iii)  Z  is  also  a  Hidden  Markov  Model:  There  is  a  finite  state  space  Markov  chain  I  on 
the  finite  set  {1, . . . ,  m},  an  i.i.d.  process  N  on  R,  and  a  function  (p:  {1, . . .  ,m}  x  R 

m}  such  that, 

Z{t 1)  =  (p{I{t),  N{t >  0. 
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What  are  the  eigenvalues  of  If  (/i,  A)  solve  the  eigenfunction  equation  Th  =  Xh  with  A  a 
possibly  complex  scalar,  and  h:  Z  — >  C,  it  follows  that  h  can  be  written  as  a  linear  combination 
of  the  functions  {sj}:  For  some  complex  scalars  {pt}  we  have, 

^  ■^€Z.  (14) 

The  form  of  T  given  in  (13)  and  the  eigpnfunetion  equation  then  give, 

m 

Linear  independence  of  the  functions  {sj}  implies  that  the  eoefheients  coincide.  Consequently, 
the  coefficients  {gk}  and  the  eigenvalue  A  are  obtained  as  the  solution  to  the  finite  matrix 
eigenvalue  problem, 

Mg  =  Xg, 

with  Mjk  :=^ij{sk)- 

The  constraints  on  and  A  in  the  FRO  model  are  expressed. 

Aft  =  ^  B'ijMjkQk,  1  <  i  <  m, 
j-k 

where  Mjk  =  conclusion,  we  obtain 

Proposition  3.2.  The  eigenvalues  ofT^*  in  the  FRO  model  correspond  with  those  of  the  m  x  m 
matrix  S*R^^(0).  If  X  ^  C  is  an  eigenvaluej  and  g  an  eigenvector, 

e*R^^g  =  Xg 

then  the  function  (11)  is  an  eigenfunction  for  T.  □ 

We  next  consider  a  general  approach  to  optimal  prediction. 


3.2  Optimal  prediction  on  a  subspace 
Suppose  that  {<l>i}  are  a  eolleetion  of  functions  in  L2(/i), 

m(0?)  •=  E[0^(Z(O)]  <00,  1  <  i  <  m. 

Our  goal  is  to  ehoase  0  =  B*  to  guarantee  the  optimal  prediction  property  ((>).  We  first  express 
the  left  hand  side  as  follows:  For  each  z,  j  =  1, . . . ,  m, 

E»~[<i)i(Z(t)(l)j(Z(t  +  l)]  =  J  <i)i(zQ)<i)j(zi)p(i(zo,zi)n{dzo)n(dzi) 


^  fdkep{4>irk)li{<i>jmi) 
k,t=\ 


(15) 


Based  on  the  notation  (9),  it  follows  that  the  equation  (6)  has  the  equivalent  matrix  formulation, 

=  R!^(\).  (16) 

We  thereby  arrive  at  a  formula  for  the  optimal  parameter. 
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Proposition  3.3,  Suppose  that  the  covariance  matrices  and  are  invertible. 

Then  the  unique  value  0/  0*  satisfying  (6)  25  defined  by  the  matrix  product f 

0*  =  (17) 

□ 


3.3  Optimal  prediction  and  L^  projection 

We  now  show  that  the  optimal  prediction  property  holds  for  an  optimal  model  obtained  under 
a  natural  optimization  criterion. 

Recall  that  possesses  a  density  with  respect  to  the  product  distribution  /x  x  /x.  Assume 
that  pP  also  possesses  a  density,  denoted  p^.  The  L2  mismatched  criterion  considered  here  is 
defined  for  any  0  by, 


=  5  j -p{^^zi)fti{dzQ)n{dzi)  (18) 

Proposition  3. 1  asserts  that  the  L2  optimal  model  exhibits  optimal  prediction  on  the  finite¬ 
dimensional  subspace  of  functions  spanned  by  the  basis.  If  {ri}  =  {mj}  are  indicator  functions 
of  a  partition  of  Z,  then  (b)  coincides  with  the  constraints  defining  the  transition  matrix  de¬ 
scribed  in  Proposition  2.2. 

Proposition  3.4.  Suppose  that  rj  =  mi  for  each  i,  that  ri  =  mi  =  1,  and  that  these  m 
functions  are  linearly  independent  in  Z/2(ax).  That  is,  the  covariance  matrix  R^{0)  is  full  rank. 
Then,  the  vector  0*  minimizes  S  if  and  only  if  the  optimal-prediction  constraints  (6)  hold  with 
{(^j}  =  {r,}.  The  unique  solution  is  expressed  uniquely, 


e*  =  {e*j.}  =  [/?’-(o)]->/?^(i)[/?’-(o)]-i  (19) 

Proof.  First  consider  the  general  setting  in  which  the  {ri}  and  {mj}  may  differ.  On  setting  the 
derivative  of  S  with  respect  to  0iojo  equal  to  zero  for  each  zq  and  Jq  we  find  that  an  optimal 
parameter  is  characterized  by  the  optimal-prediction  constraints, 


E[ri{Z{t)mj{Z{t+l)]  = 


/ 

/ 


ri{zo)mj{zi)p{zo,  zi)n{dzo)n{dzi) 
ri{zo)Tnj{zi)p»-{zo,zi)n{dzo)p{dzi) 


i,j  =  l,...,m 


(20) 


Under  the  assumption  that  ri  and  mi  are  identically  equal  to  unity,  the  constraints  (20)  imply 
that, 

1  =  E[ri(Z(()mi(Z((+ 1)]  =  j  p».{zo,  zi)p{dzo)p{dzi) 
so  that  has  total  mass  one. 

To  complete  the  proof  it  is  necessary  to  demonstrate  that  the  bivariate  distribution  has 
equal  marginals  ^h-i  =  with 
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